-- Logs begin at Sat 2022-09-10 23:11:52 UTC, end at Sun 2022-09-11 00:22:08 UTC. -- Sep 10 23:40:42.788784 dv-cinder-852009-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 10 23:40:42.797499 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 10 23:40:42.799539 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Sep 10 23:40:42 2022] *** Sep 10 23:40:42.799539 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 10 23:40:42.799539 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 10 23:40:42.799539 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: nodename: dv-cinder-852009-6-iscsi Sep 10 23:40:42.799539 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: machine: x86_64 Sep 10 23:40:42.799539 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: clock source: unix Sep 10 23:40:42.799539 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: pcre jit disabled Sep 10 23:40:42.799539 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: detected number of CPU cores: 4 Sep 10 23:40:42.799539 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: current working directory: / Sep 10 23:40:42.799539 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: detected binary path: /usr/bin/uwsgi-core Sep 10 23:40:42.799539 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: your processes number limit is 47680 Sep 10 23:40:42.800130 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: your memory page size is 4096 bytes Sep 10 23:40:42.800130 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: detected max file descriptor number: 2048 Sep 10 23:40:42.800130 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: lock engine: pthread robust mutexes Sep 10 23:40:42.800130 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: thunder lock: enabled Sep 10 23:40:42.800130 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 10 23:40:42.800130 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 10 23:40:42.838261 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: Python main interpreter initialized at 0x563712ee69c0 Sep 10 23:40:42.838261 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: python threads support enabled Sep 10 23:40:42.838261 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: your server socket listen backlog is limited to 100 connections Sep 10 23:40:42.838261 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: your mercy for graceful operations on workers is 90 seconds Sep 10 23:40:42.838606 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: mapped 403077 bytes (393 KB) for 2 cores Sep 10 23:40:42.838708 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: *** Operational MODE: preforking *** Sep 10 23:40:42.838961 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: *** uWSGI is running in multiple interpreter mode *** Sep 10 23:40:42.838961 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: spawned uWSGI master process (pid: 81560) Sep 10 23:40:42.839158 dv-cinder-852009-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 10 23:40:42.842923 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: spawned uWSGI worker 1 (pid: 81573, cores: 1) Sep 10 23:40:42.842923 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: spawned uWSGI worker 2 (pid: 81574, cores: 1) Sep 10 23:40:42.842923 dv-cinder-852009-6-iscsi devstack@keystone.service[81560]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 10 23:40:47.173793 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 10 23:40:47.173793 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: from cryptography.utils import int_from_bytes Sep 10 23:40:47.174593 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 10 23:40:47.174593 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: from cryptography.utils import int_from_bytes Sep 10 23:40:47.175051 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 10 23:40:47.175051 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: from cryptography.utils import int_from_bytes Sep 10 23:40:47.175170 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 10 23:40:47.175170 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: from cryptography.utils import int_from_bytes Sep 10 23:40:47.234054 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 10 23:40:47.234240 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 10 23:40:47.234240 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 10 23:40:47.234409 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 10 23:40:47.234483 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] command line args: [] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 10 23:40:47.234551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] command line args: [] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 10 23:40:47.234693 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 10 23:40:47.234787 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 10 23:40:47.234860 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ================================================================================ {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 10 23:40:47.234938 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ================================================================================ {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 10 23:40:47.235094 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] admin_token = **** {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.235173 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] admin_token = **** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.235337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] config_dir = [] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.235421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] config_dir = [] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.235521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.235610 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.235681 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] config_source = [] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.235755 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] config_source = [] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.235859 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] debug = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.235936 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] debug = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.236052 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.236301 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.236452 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.236527 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.236594 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.236679 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.236837 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.236926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.237057 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.237146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.237278 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] list_limit = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.237368 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] list_limit = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.237534 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] log_config_append = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.237692 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] log_config_append = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.237871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.237976 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.238077 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] log_dir = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.238181 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] log_dir = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.238277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] log_file = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.238411 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] log_file = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.238524 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.238813 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.238895 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.239018 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.239097 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.239218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.239298 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.239417 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.239501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.239619 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.239699 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.239823 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.239904 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.240020 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.240102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.240225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.240300 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.240420 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.240495 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.240620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.240700 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.240821 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.240895 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.241015 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.241090 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.241214 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.241294 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.241414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.241489 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.241603 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.241705 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.241794 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.241872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] publish_errors = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.241995 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] publish_errors = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.242100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.242184 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.242327 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.242409 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.242523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.242639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.242762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.242862 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.242993 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.243071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.243223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] standard_threads = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.243307 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] standard_threads = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.243450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.243526 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.243685 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.243763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.243844 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.243921 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.244015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] use_journal = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.244097 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] use_journal = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.244174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] use_json = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.244249 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] use_json = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.244327 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] use_stderr = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.244398 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] use_stderr = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.244498 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] use_syslog = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.244575 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] use_syslog = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.244724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.244835 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 23:40:47.244969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.245048 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.245212 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.245302 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.245475 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.245557 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.245704 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.245782 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.245945 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.246022 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.246180 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.246276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.246422 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.246506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.246653 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.246761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.246919 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.247004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.247160 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.247238 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.247404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.247480 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.247635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.247719 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.247874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.247957 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.248104 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.248191 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.248365 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.248512 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.248611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.248791 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.248870 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.249040 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.249123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.249301 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.249389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.249555 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.249634 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.249804 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.249881 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.250044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.250128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.250299 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.250382 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.250547 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.250631 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.250822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.250904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.251054 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.251137 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.251283 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.251360 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.251502 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.251575 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.251729 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.251805 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.251964 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.252083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.252206 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.252319 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.252431 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.252543 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.252661 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.252774 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.252895 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.253014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.253134 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.253247 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.253365 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.253478 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.253736 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.253939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.254055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.254177 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.254291 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.254412 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.254525 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.254637 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.254771 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.254879 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.254977 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.255079 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.255183 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.255278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.255372 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.255466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.255602 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.255711 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.255807 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.255921 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.256043 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.256157 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.256271 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.256392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.256506 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.256618 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.256731 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.256843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.256964 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.257087 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.257200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.257314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.257439 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.257554 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.257668 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.257781 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.257903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.258018 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.258130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.258242 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.258356 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.258475 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.258596 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.258734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.258852 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.258965 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.259078 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.259191 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.259314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.259431 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.259551 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.259664 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.259775 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.259895 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.260016 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.260131 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.260256 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.260382 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.260504 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.260674 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.260797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.260904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.261006 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.261101 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.261196 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.261291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.261385 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.261488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.261585 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.261679 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.261780 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.261780 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.261974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.262079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.262174 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.262268 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.262370 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.262479 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.262575 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.266409 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.266521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.266722 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.266801 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.266872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.266941 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.267024 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.267097 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.267167 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.267235 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.267304 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.267385 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.267463 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.267541 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.267612 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.267681 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.267751 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.267820 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.267889 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.267966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.268037 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.268106 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.268175 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.268243 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.268312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.268388 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.268459 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.268528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.268602 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.268673 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.268742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.268819 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.268891 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.268960 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.269035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.269035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.269161 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.269243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.269316 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.269385 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.269454 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.269522 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.269591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.269667 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.269738 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.269808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.269876 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.269945 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.270014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.270093 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.270170 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.270240 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.270309 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.270377 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.270445 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.270525 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.270597 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.270707 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.270794 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.270873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.270993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.271077 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] auth.external = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.271149 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.271245 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.271324 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.271396 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.271467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.271544 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.271622 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] auth.external = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.271700 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] auth.password = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.271772 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.271850 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] auth.token = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.271921 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.271992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.272062 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.272139 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.272210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] auth.password = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.272280 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.272349 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] auth.token = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.272418 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.272495 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.272572 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.272643 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.272760 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.272842 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.272913 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.272992 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.273064 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] credential.caching = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.273134 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.273207 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.273277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.273347 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.273506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.273600 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.273675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] credential.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.273795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.273871 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.273942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.274024 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.274103 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.274175 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.274245 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.274314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.274384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.274462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.274534 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.274604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.274696 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.274802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.274881 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.275026 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.275109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.275265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.275344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.275518 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.275596 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.275802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.275890 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.276032 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] federation.caching = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.276109 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.276265 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.276343 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.276506 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.276586 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.276747 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.276832 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.276973 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.277059 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] federation.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.277214 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.277294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.277446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.277524 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.277692 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.277769 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.277928 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.278017 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.278171 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.278250 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.278409 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.278490 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.278706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.278791 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.278918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.caching = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.278995 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.279160 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.279238 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.279395 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.279474 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.280142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.280231 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.280378 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.280516 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.280626 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.280770 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.281020 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.281096 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.281257 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.281335 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.281495 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.281575 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.281738 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.281815 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.281966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.282043 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.282263 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.282378 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.282477 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.282591 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.282710 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.282839 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.282963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.283078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.283192 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.283305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.283418 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.283541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.283657 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.283771 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.283885 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.283998 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.284111 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.284233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.284348 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.284461 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.284576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.284690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.284803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.284925 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.285041 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.285154 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.285268 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.285387 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.285501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.285629 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.285744 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.285858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.285972 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.286098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.286221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.286344 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.286459 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.286574 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.286706 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.286823 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.286937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.287063 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.287180 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.287293 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.287406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.287520 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.287639 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.287763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.287877 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.287991 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.288104 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.288217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.288331 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.288452 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.288567 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.288681 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.288795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.288939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.289043 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.289148 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.289245 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.289340 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.289435 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.289530 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.289625 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.289728 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.289825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.289926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.290022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.290116 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.290211 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.290382 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.290488 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.290627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.290780 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.290879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.290974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.296049 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.296202 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.296336 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.296468 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.296600 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.296733 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.296877 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.297011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.297144 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.297275 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.297405 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.297536 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.297675 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.297807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.297948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.298079 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.298211 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.298342 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.298481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.298620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.298777 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.298910 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.299040 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.299181 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.299316 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.299446 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.299577 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.299664 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.299753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.299835 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.299909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.299980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.300051 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.300121 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.300192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.300271 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.300343 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.300414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.300414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.300541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.300611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.300689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.300689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.300818 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.300895 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.300967 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.301037 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.301115 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.301187 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.301258 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.301323 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.301381 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.301439 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.301505 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.301565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.301622 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.301680 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.301737 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.301794 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.301861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.301921 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.302017 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.302199 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.302266 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.302324 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.302457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.302521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.302580 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.302637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.302785 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.302865 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.302939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.303002 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.303067 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.303127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.303185 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.303242 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] resource.caching = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.303307 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.303366 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.303424 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.303482 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.303539 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.303610 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.303676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.303736 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.303794 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.303851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.303908 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] resource.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.303965 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.304031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.304090 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.304148 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.304251 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.304328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.304388 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.304455 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.304515 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] role.caching = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.304574 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.304632 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] role.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.304689 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.304747 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.304814 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.304877 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.304936 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.304994 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.305053 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.305111 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.305177 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] role.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.305237 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.305296 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] role.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.305353 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.305411 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.305469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.305535 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.306378 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.306523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.306652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.306753 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.306840 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.306915 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.306986 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.307056 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.307126 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.307196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.307274 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.307346 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.307416 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.307486 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.307556 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.307634 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.307706 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.307783 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.307855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.307932 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.308004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.308082 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.308154 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.308231 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.308302 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.308373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.308443 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.308521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.308593 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.308664 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.308734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.308892 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.308972 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.309115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.309195 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.309346 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.309424 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.309575 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.309655 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.309819 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.309900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.310066 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.310145 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.310305 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.310382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.310536 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.310614 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.310823 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.311022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.311171 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.311289 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.311417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] token.caching = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.311533 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.311647 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.311761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.311876 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.311989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.312112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.312227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.312348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.312463 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.312577 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.312691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.312814 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] token.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.312931 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.313045 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.313159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.313272 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.313398 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.313513 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.313662 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.313776 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.313891 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.314003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.314124 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.314240 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.314354 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.314467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.314581 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.314707 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.314836 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.314955 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.315078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.315201 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.315316 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.315316 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.315660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.315774 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.315898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.connection = **** {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.316015 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.316129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.316129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.316342 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.316456 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.316582 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.316707 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.316832 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.316948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.317063 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.317177 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.317290 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.connection = **** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.317412 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.317528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.317641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.317755 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.317869 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.317982 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.318172 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.318290 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.318405 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.318518 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.318631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.318762 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.318890 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.319008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.319122 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.319237 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.319351 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.319465 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.319588 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.319703 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.319817 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.319931 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.320048 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.320048 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81573) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 10 23:40:47.320273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.320273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.320273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.320587 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.320700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.320927 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.321166 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 23:40:47.321338 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 10 23:40:47.354875 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.355038 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.355110 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.355202 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.355315 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.355399 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.364263 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.364437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.392138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.392335 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.392426 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.392565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.452001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.452151 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.501005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.501231 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.501731 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.501971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.657396 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.657637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.657821 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.658038 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.658250 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.658365 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.658461 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.658601 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.658854 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.659074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.659292 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.659506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.681938 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.682052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.682187 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.682266 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.682367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.682511 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.682592 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.683001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.683091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.683166 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.683296 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.683381 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.683610 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.683686 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.700546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.701011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.713846 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.714020 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.714245 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.714456 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.714540 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.714692 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.714776 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.714907 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.714988 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.715115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81573) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.715210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.715364 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.715582 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.715798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 23:40:47.736836 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.737131 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 10 23:40:47.737367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.737450 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.737566 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 10 23:40:47.737648 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 10 23:40:47.737837 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.737975 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.738095 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.738176 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 10 23:40:47.738348 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.738429 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.738599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.738757 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.739031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.739291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.748239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.748546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.748818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.749101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.749204 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.749402 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.749677 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.749964 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.756030 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.757140 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.758964 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.759245 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.759522 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.759799 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.760073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.760155 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.760330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.760409 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.760597 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.760680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.760870 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.760951 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.761134 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.761274 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.761407 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.761549 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.761668 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.761751 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.761941 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.762213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.762487 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.762785 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.785283 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.786751 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.787012 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.787096 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.788500 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.788798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.791648 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.791933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.792211 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.793669 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.793958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.794243 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.798569 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.798870 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.800945 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.801234 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.804473 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.804766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.805054 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.805341 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.805622 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.806155 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.806437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.806725 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.807272 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.807572 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.807869 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.808155 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.808432 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.808710 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.808983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.809253 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.822190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.822464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.825073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.825347 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.826319 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.829774 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.830070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.830365 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.832984 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.833284 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.834565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.837481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.837833 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.840759 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.841057 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.841354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.841636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.841914 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.842276 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.842562 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.844065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.844370 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.844730 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.844992 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.845285 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.845570 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.859228 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.859519 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.859785 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.860048 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.862227 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.862515 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.862811 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.863080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.866880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.868408 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.869915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.870075 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.870171 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.871697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.873103 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.873224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.873448 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.873669 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.873752 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.873995 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.874385 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.876513 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.876796 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.877080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.877364 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.877765 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.885564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.885844 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.886119 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.886395 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.886794 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.887076 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.887352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.889769 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.890107 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.890455 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.890807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.891126 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.891406 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.891686 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.903214 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.906334 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.907261 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.909291 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.910225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.910799 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.912243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.912526 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.912916 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.913256 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.914809 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.916270 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.916557 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.916956 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.919275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.921399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.924983 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.924983 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.924983 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.925848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.927552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.928886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.932090 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.932490 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.935474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.935474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.935652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.935652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.935652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.941448 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81573) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.941673 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.941758 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.941972 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.942056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.942290 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.942376 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.942576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.942874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.943161 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.943434 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.943698 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.943973 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.944249 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.944509 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.944918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.945245 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81573) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.949329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 23:40:47.949619 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.949920 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.950188 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.950468 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.950770 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.951060 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.951347 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.951663 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.951998 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.952238 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.952504 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.952917 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.953325 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 23:40:47.979943 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x563712ee69c0 pid: 81573 (default app) Sep 10 23:40:47.983828 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b0958-ac36-4afe-837d-825e45cb3d76 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:40:47.984105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b0958-ac36-4afe-837d-825e45cb3d76 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:40:47.984373 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b0958-ac36-4afe-837d-825e45cb3d76 None None] PATH_INFO: `/v3` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:40:47.988512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Sat Sep 10 23:40:47 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:40:47.988790 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x563712ee69c0 pid: 81574 (default app) Sep 10 23:40:55.720779 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf93f0-2471-4862-9f13-43e1eb2a9717 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:40:55.721283 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf93f0-2471-4862-9f13-43e1eb2a9717 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:40:55.721363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf93f0-2471-4862-9f13-43e1eb2a9717 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:40:55.722611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 2/2] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:40:55.730131 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b7274-8f3d-4b4d-adc3-a112491c75a0 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:40:55.730584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b7274-8f3d-4b4d-adc3-a112491c75a0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:40:55.730693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b7274-8f3d-4b4d-adc3-a112491c75a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:40:55.802136 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: INFO dbcounter [None req-177b7274-8f3d-4b4d-adc3-a112491c75a0 None None] Registered counter for database keystone Sep 10 23:40:55.820962 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG oslo_db.sqlalchemy.engines [None req-177b7274-8f3d-4b4d-adc3-a112491c75a0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81574) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 10 23:40:55.827926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writer thread running {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 10 23:40:56.012327 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG passlib.handlers.bcrypt [None req-177b7274-8f3d-4b4d-adc3-a112491c75a0 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81574) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 10 23:40:56.012641 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG passlib.handlers.bcrypt [None req-177b7274-8f3d-4b4d-adc3-a112491c75a0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81574) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 10 23:40:56.070982 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-177b7274-8f3d-4b4d-adc3-a112491c75a0 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:40:56.072581 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-177b7274-8f3d-4b4d-adc3-a112491c75a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:40:56.201349 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1/3] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:40:55 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 10 23:40:56.663595 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf93f0-2471-4862-9f13-43e1eb2a9717 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:40:56.734308 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: INFO dbcounter [None req-a6cf93f0-2471-4862-9f13-43e1eb2a9717 None None] Registered counter for database keystone Sep 10 23:40:56.755814 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG oslo_db.sqlalchemy.engines [None req-a6cf93f0-2471-4862-9f13-43e1eb2a9717 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81573) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 10 23:40:56.761919 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writer thread running {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 10 23:40:56.880332 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf93f0-2471-4862-9f13-43e1eb2a9717 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:40:56.884546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbff218-2529-43cc-aab3-bd2863963a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:40:56.885174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbff218-2529-43cc-aab3-bd2863963a57 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:40:56.885451 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbff218-2529-43cc-aab3-bd2863963a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:40:56.885718 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbff218-2529-43cc-aab3-bd2863963a57 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:40:56.894393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbff218-2529-43cc-aab3-bd2863963a57 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:40:56.945773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: WARNING keystone.common.rbac_enforcer.enforcer [None req-dbbff218-2529-43cc-aab3-bd2863963a57 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 10 23:40:56.946057 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbff218-2529-43cc-aab3-bd2863963a57 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:40:56.955428 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 3/4] 10.222.0.17 () {58 vars in 1324 bytes} [Sat Sep 10 23:40:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 23:40:56.961417 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b7274-8f3d-4b4d-adc3-a112491c75a0 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:40:56.983954 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b7274-8f3d-4b4d-adc3-a112491c75a0 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:40:56.988553 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23589c7-0102-4d5f-84c3-07b0cafd1b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:40:56.989812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23589c7-0102-4d5f-84c3-07b0cafd1b23 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:40:56.990097 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23589c7-0102-4d5f-84c3-07b0cafd1b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:40:56.990372 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23589c7-0102-4d5f-84c3-07b0cafd1b23 admin admin] PATH_INFO: `/v3/projects/ab88f917c063407982ac832bef946248` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:40:56.991834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23589c7-0102-4d5f-84c3-07b0cafd1b23 admin admin] RBAC: Authorizing `identity:get_project(project_id=ab88f917c063407982ac832bef946248)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:40:57.047000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c23589c7-0102-4d5f-84c3-07b0cafd1b23 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 10 23:40:57.047337 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23589c7-0102-4d5f-84c3-07b0cafd1b23 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:40:57.051941 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 2/5] 10.222.0.17 () {56 vars in 1207 bytes} [Sat Sep 10 23:40:56 2022] GET /identity/v3/projects/ab88f917c063407982ac832bef946248 => generated 344 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:40:59.181357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273f08b-45da-4f21-951a-dda243dac22e None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:40:59.181872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273f08b-45da-4f21-951a-dda243dac22e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:40:59.181958 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273f08b-45da-4f21-951a-dda243dac22e None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:40:59.186635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 4/6] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:40:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:40:59.191997 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf56c34-d0fe-48da-b56a-69bb2f7a2d96 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:40:59.192404 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf56c34-d0fe-48da-b56a-69bb2f7a2d96 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:40:59.192553 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf56c34-d0fe-48da-b56a-69bb2f7a2d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:40:59.212808 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-edf56c34-d0fe-48da-b56a-69bb2f7a2d96 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:40:59.214310 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-edf56c34-d0fe-48da-b56a-69bb2f7a2d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:40:59.222593 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 3/7] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:40:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 10 23:40:59.490773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273f08b-45da-4f21-951a-dda243dac22e None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:40:59.503516 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273f08b-45da-4f21-951a-dda243dac22e None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:40:59.507646 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fae0ec-d6bb-405f-8b8e-cd9ce7e7fb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:40:59.508236 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fae0ec-d6bb-405f-8b8e-cd9ce7e7fb4f admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:40:59.508540 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fae0ec-d6bb-405f-8b8e-cd9ce7e7fb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:40:59.508809 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fae0ec-d6bb-405f-8b8e-cd9ce7e7fb4f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:40:59.516868 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fae0ec-d6bb-405f-8b8e-cd9ce7e7fb4f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:40:59.518630 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fae0ec-d6bb-405f-8b8e-cd9ce7e7fb4f admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:40:59.527449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 5/8] 10.222.0.17 () {58 vars in 1324 bytes} [Sat Sep 10 23:40:59 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 23:40:59.533417 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf56c34-d0fe-48da-b56a-69bb2f7a2d96 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:40:59.542386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf56c34-d0fe-48da-b56a-69bb2f7a2d96 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:40:59.547320 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2444c39-461f-4bdf-9bde-18d623c41d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:40:59.547938 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2444c39-461f-4bdf-9bde-18d623c41d77 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:40:59.548217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2444c39-461f-4bdf-9bde-18d623c41d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:40:59.548494 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2444c39-461f-4bdf-9bde-18d623c41d77 admin admin] PATH_INFO: `/v3/users/5243f89b8bf44f868ec5406d662b5cd9` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:40:59.549790 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2444c39-461f-4bdf-9bde-18d623c41d77 admin admin] RBAC: Authorizing `identity:get_user(user_id=5243f89b8bf44f868ec5406d662b5cd9)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:40:59.551700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2444c39-461f-4bdf-9bde-18d623c41d77 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:40:59.553219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 4/9] 10.222.0.17 () {56 vars in 1198 bytes} [Sat Sep 10 23:40:59 2022] GET /identity/v3/users/5243f89b8bf44f868ec5406d662b5cd9 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:41:01.814490 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7d9e1-1b48-485c-b8eb-bb9da33ccfa9 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:01.815110 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7d9e1-1b48-485c-b8eb-bb9da33ccfa9 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:01.815110 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7d9e1-1b48-485c-b8eb-bb9da33ccfa9 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:01.816368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 6/10] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:41:01.821579 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05234dfa-61b4-4115-9d50-8808e84a7ec8 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:01.822004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05234dfa-61b4-4115-9d50-8808e84a7ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:01.822125 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05234dfa-61b4-4115-9d50-8808e84a7ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:01.832772 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb56efb-c1a7-4cba-b7d1-d617102fb48d None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:01.833053 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb56efb-c1a7-4cba-b7d1-d617102fb48d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:01.833322 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb56efb-c1a7-4cba-b7d1-d617102fb48d None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:01.834612 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 7/11] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:41:01.840557 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-05234dfa-61b4-4115-9d50-8808e84a7ec8 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:01.841231 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6895d3d-be11-4a31-ae5d-ec9da4015aa6 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:01.841728 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6895d3d-be11-4a31-ae5d-ec9da4015aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:01.841835 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-05234dfa-61b4-4115-9d50-8808e84a7ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:01.842243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6895d3d-be11-4a31-ae5d-ec9da4015aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:01.885790 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG passlib.handlers.bcrypt [None req-a6895d3d-be11-4a31-ae5d-ec9da4015aa6 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81573) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 10 23:41:01.886172 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG passlib.handlers.bcrypt [None req-a6895d3d-be11-4a31-ae5d-ec9da4015aa6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81573) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 10 23:41:01.922734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 5/12] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:01.936512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a6895d3d-be11-4a31-ae5d-ec9da4015aa6 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:01.938113 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a6895d3d-be11-4a31-ae5d-ec9da4015aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:02.041590 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 8/13] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 23:41:02.188525 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05234dfa-61b4-4115-9d50-8808e84a7ec8 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:02.213382 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:02.292110 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05234dfa-61b4-4115-9d50-8808e84a7ec8 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:02.313627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6895d3d-be11-4a31-ae5d-ec9da4015aa6 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:02.329070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71eee00-0e1f-4aca-9e4b-b9b819f907d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:02.329704 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71eee00-0e1f-4aca-9e4b-b9b819f907d6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:02.329986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71eee00-0e1f-4aca-9e4b-b9b819f907d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:02.330261 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71eee00-0e1f-4aca-9e4b-b9b819f907d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:02.341351 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71eee00-0e1f-4aca-9e4b-b9b819f907d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:02.350745 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-f71eee00-0e1f-4aca-9e4b-b9b819f907d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:41:02.350745 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:02.350745 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:02.350745 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:02.350745 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:02.350745 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:02.350745 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:02.350745 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:02.350745 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:02.350745 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:02.350745 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:02.350745 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:02.350745 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:02.350745 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:02.350745 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:02.350745 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:02.351693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:02.351693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:02.351693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:02.351693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:02.351693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:02.351693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:02.351693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:02.351693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:02.351693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:02.351693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:02.351693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:02.351693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:02.351693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:02.351693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:02.351693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:02.351693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:02.352549 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:02.352549 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:02.352549 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:02.352549 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:02.352549 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:02.352549 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:02.352549 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:02.352549 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:02.352549 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:02.352549 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:02.352549 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:41:02.352549 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:41:02.352549 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 6/14] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:41:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:02.358316 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71eee00-0e1f-4aca-9e4b-b9b819f907d6 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:02.414724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6895d3d-be11-4a31-ae5d-ec9da4015aa6 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:02.416993 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:02.439951 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:02.452458 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd82ad-9896-4948-bfc3-2659cc317ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:02.453321 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd82ad-9896-4948-bfc3-2659cc317ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:02.453600 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd82ad-9896-4948-bfc3-2659cc317ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:02.453874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd82ad-9896-4948-bfc3-2659cc317ca8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:02.456808 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71eee00-0e1f-4aca-9e4b-b9b819f907d6 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:02.490905 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacdbe2-f2b3-4f02-b54e-26e71cd6b694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:02.491504 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacdbe2-f2b3-4f02-b54e-26e71cd6b694 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:02.491786 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacdbe2-f2b3-4f02-b54e-26e71cd6b694 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:02.492056 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacdbe2-f2b3-4f02-b54e-26e71cd6b694 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:02.492721 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bacdbe2-f2b3-4f02-b54e-26e71cd6b694 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:02.493023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bacdbe2-f2b3-4f02-b54e-26e71cd6b694 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:02.494422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bacdbe2-f2b3-4f02-b54e-26e71cd6b694 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:02.506241 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 7/15] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:41:02.511957 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacdbe2-f2b3-4f02-b54e-26e71cd6b694 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:02.521837 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd82ad-9896-4948-bfc3-2659cc317ca8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:02.523384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd82ad-9896-4948-bfc3-2659cc317ca8 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:02.611159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacdbe2-f2b3-4f02-b54e-26e71cd6b694 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:02.620269 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 9/16] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:41:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:41:02.626866 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd82ad-9896-4948-bfc3-2659cc317ca8 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:02.629506 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:02.644043 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b7c7b-57b3-4c8f-b49c-e9f3f37cbd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:02.644842 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b7c7b-57b3-4c8f-b49c-e9f3f37cbd22 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:02.645125 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b7c7b-57b3-4c8f-b49c-e9f3f37cbd22 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:02.645397 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b7c7b-57b3-4c8f-b49c-e9f3f37cbd22 None admin] PATH_INFO: `/v3/users/5243f89b8bf44f868ec5406d662b5cd9` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:02.646527 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b7c7b-57b3-4c8f-b49c-e9f3f37cbd22 None admin] RBAC: Authorizing `identity:get_user(user_id=5243f89b8bf44f868ec5406d662b5cd9)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:02.648007 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b7c7b-57b3-4c8f-b49c-e9f3f37cbd22 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:02.649559 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 8/17] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:02 2022] GET /identity/v3/users/5243f89b8bf44f868ec5406d662b5cd9 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:41:02.655013 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b7c7b-57b3-4c8f-b49c-e9f3f37cbd22 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:02.669321 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:02.724511 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd82ad-9896-4948-bfc3-2659cc317ca8 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:02.753621 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b7c7b-57b3-4c8f-b49c-e9f3f37cbd22 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:02.759047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e9426-16c9-474d-aca2-7e8b8b43a59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:02.759888 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e9426-16c9-474d-aca2-7e8b8b43a59f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:02.760159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e9426-16c9-474d-aca2-7e8b8b43a59f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:02.760421 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e9426-16c9-474d-aca2-7e8b8b43a59f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:02.778421 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e9426-16c9-474d-aca2-7e8b8b43a59f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:02.779929 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e9426-16c9-474d-aca2-7e8b8b43a59f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:02.786596 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3cc14-3ab0-4b08-bb48-6e1582b51a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:02.787245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3cc14-3ab0-4b08-bb48-6e1582b51a96 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:02.787521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3cc14-3ab0-4b08-bb48-6e1582b51a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:02.787789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3cc14-3ab0-4b08-bb48-6e1582b51a96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:02.788888 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc3cc14-3ab0-4b08-bb48-6e1582b51a96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:02.790354 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc3cc14-3ab0-4b08-bb48-6e1582b51a96 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:02.791919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 9/18] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:41:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:41:02.798288 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3cc14-3ab0-4b08-bb48-6e1582b51a96 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:02.798800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-2c3e9426-16c9-474d-aca2-7e8b8b43a59f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:41:02.798800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:02.798800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:02.798800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:02.798800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:02.798800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:02.798800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:02.798800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:02.798800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:02.798800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:02.798800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:02.798800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:02.798800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:41:02.798800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:41:02.798800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:41:02.798800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:41:02.799742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:02.799742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:02.799742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:02.799742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:02.799742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:02.799742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:02.799742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:02.799742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:02.799742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:02.799742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:02.799742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:02.799742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:02.799742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:02.799742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:02.799742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:02.799742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:02.799742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:41:02.800543 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:41:02.800543 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:41:02.800543 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:41:02.800543 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 10/19] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:41:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:41:02.805517 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e9426-16c9-474d-aca2-7e8b8b43a59f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:02.862489 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:02.879549 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:02.898263 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3cc14-3ab0-4b08-bb48-6e1582b51a96 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:02.907130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e9426-16c9-474d-aca2-7e8b8b43a59f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:02.932199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106ed3b-405b-4ec8-adcd-c1ea08c2ebad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:02.932986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106ed3b-405b-4ec8-adcd-c1ea08c2ebad None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:02.933260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106ed3b-405b-4ec8-adcd-c1ea08c2ebad None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:02.933530 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106ed3b-405b-4ec8-adcd-c1ea08c2ebad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:02.934244 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106ed3b-405b-4ec8-adcd-c1ea08c2ebad None admin] RBAC: Adding query filter params (role.id=9dce48a2b53c482d931dcd064b07744e, scope.domain.id=default, user.id=5243f89b8bf44f868ec5406d662b5cd9) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:02.934550 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106ed3b-405b-4ec8-adcd-c1ea08c2ebad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:02.936660 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106ed3b-405b-4ec8-adcd-c1ea08c2ebad None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:02.942063 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fa383-6344-4dc6-9ebd-b294c86378a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:02.942682 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fa383-6344-4dc6-9ebd-b294c86378a4 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:02.942988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fa383-6344-4dc6-9ebd-b294c86378a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:02.943249 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 10/20] 10.222.0.17 () {56 vars in 1322 bytes} [Sat Sep 10 23:41:02 2022] GET /identity/v3/role_assignments?user.id=5243f89b8bf44f868ec5406d662b5cd9&scope.domain.id=default&role.id=9dce48a2b53c482d931dcd064b07744e => generated 233 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:41:02.943349 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fa383-6344-4dc6-9ebd-b294c86378a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:02.943913 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fa383-6344-4dc6-9ebd-b294c86378a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:02.944198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fa383-6344-4dc6-9ebd-b294c86378a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:02.945578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fa383-6344-4dc6-9ebd-b294c86378a4 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:02.969670 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 11/21] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:41:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:41:05.656143 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85e375-03c8-47f0-81b0-cdce765ca552 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:05.656639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85e375-03c8-47f0-81b0-cdce765ca552 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:05.656639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85e375-03c8-47f0-81b0-cdce765ca552 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:05.658363 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 11/22] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:05.664320 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5bf120-1ccd-43ec-8c73-2ce01d72caca None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:05.664828 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5bf120-1ccd-43ec-8c73-2ce01d72caca None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:05.664828 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5bf120-1ccd-43ec-8c73-2ce01d72caca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:05.695760 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-3e5bf120-1ccd-43ec-8c73-2ce01d72caca None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:05.696977 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3e5bf120-1ccd-43ec-8c73-2ce01d72caca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:05.785739 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc760079-6cae-42cc-9041-898b91452609 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:05.786079 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc760079-6cae-42cc-9041-898b91452609 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:05.786293 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc760079-6cae-42cc-9041-898b91452609 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:05.788140 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 12/23] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:05.788731 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 12/24] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:05.793950 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05d6a0-bdc9-4870-b4c1-db1cf19ba917 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:05.794234 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05d6a0-bdc9-4870-b4c1-db1cf19ba917 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:05.794509 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05d6a0-bdc9-4870-b4c1-db1cf19ba917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:05.829773 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5b05d6a0-bdc9-4870-b4c1-db1cf19ba917 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:05.831033 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5b05d6a0-bdc9-4870-b4c1-db1cf19ba917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:05.849036 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893b694-0b8b-42bd-bc0c-362231cb9d5c None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:05.849624 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893b694-0b8b-42bd-bc0c-362231cb9d5c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:05.850114 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893b694-0b8b-42bd-bc0c-362231cb9d5c None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:05.851666 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 13/25] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:05.856927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f5870-8f47-4e22-9b6e-b70efac79e33 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:05.857618 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f5870-8f47-4e22-9b6e-b70efac79e33 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:05.858099 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f5870-8f47-4e22-9b6e-b70efac79e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:05.895844 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-326f5870-8f47-4e22-9b6e-b70efac79e33 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:05.897449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-326f5870-8f47-4e22-9b6e-b70efac79e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:05.957740 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 13/26] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:05.975098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:06.008024 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 14/27] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 23:41:06.075441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f064df9-8d1c-43ef-8ea1-109c7051c2c9 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:06.075748 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f064df9-8d1c-43ef-8ea1-109c7051c2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:06.075996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f064df9-8d1c-43ef-8ea1-109c7051c2c9 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:06.077899 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 14/28] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:06.083545 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bcd3a-dba5-4646-94ea-a2b7cf3fcfee None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:06.083881 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bcd3a-dba5-4646-94ea-a2b7cf3fcfee None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:06.084163 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bcd3a-dba5-4646-94ea-a2b7cf3fcfee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:06.110235 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-f64bcd3a-dba5-4646-94ea-a2b7cf3fcfee None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:06.111483 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f64bcd3a-dba5-4646-94ea-a2b7cf3fcfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:06.159050 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f064df9-8d1c-43ef-8ea1-109c7051c2c9 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:06.237275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 15/29] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:06.303279 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:06.344588 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f064df9-8d1c-43ef-8ea1-109c7051c2c9 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:06.366374 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bcd3a-dba5-4646-94ea-a2b7cf3fcfee None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:06.412512 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8277b-9123-431b-87c6-5a377d1939bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:06.413240 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8277b-9123-431b-87c6-5a377d1939bb None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:06.413550 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8277b-9123-431b-87c6-5a377d1939bb None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:06.413826 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8277b-9123-431b-87c6-5a377d1939bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:06.448475 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a8277b-9123-431b-87c6-5a377d1939bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:06.463481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-f8a8277b-9123-431b-87c6-5a377d1939bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:41:06.463481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:06.463481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:06.463481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:06.463481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:06.463481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:06.463481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:06.463481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:06.463481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:06.463481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:06.463481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:06.463481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:06.463481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:06.463481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:06.463481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:06.463481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:06.464386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:06.464386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:06.464386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:06.464386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:06.464386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:06.464386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:06.464386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:06.464386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:06.464386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:06.464386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:06.464386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:06.464386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:06.464386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:06.464386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:06.464386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:06.464386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:06.465227 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:06.465227 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:06.465227 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:06.465227 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:06.465227 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:06.465227 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:06.465227 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:06.465227 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:06.465227 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:06.465227 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:06.465227 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:41:06.465227 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:41:06.465227 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 15/30] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:41:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:41:06.474489 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8277b-9123-431b-87c6-5a377d1939bb None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:06.634525 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bcd3a-dba5-4646-94ea-a2b7cf3fcfee None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:06.651010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:06.688441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8277b-9123-431b-87c6-5a377d1939bb None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:06.735409 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6d9ba-161a-49a9-aade-3e2ecf23ad6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:06.738689 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6d9ba-161a-49a9-aade-3e2ecf23ad6b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:06.738689 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6d9ba-161a-49a9-aade-3e2ecf23ad6b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:06.738689 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6d9ba-161a-49a9-aade-3e2ecf23ad6b None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:06.738689 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa6d9ba-161a-49a9-aade-3e2ecf23ad6b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:06.738689 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa6d9ba-161a-49a9-aade-3e2ecf23ad6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:06.742682 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa6d9ba-161a-49a9-aade-3e2ecf23ad6b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:06.743504 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:06.752081 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d2715-3c91-4acd-b65c-f136c73e8424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:06.753254 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d2715-3c91-4acd-b65c-f136c73e8424 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:06.753649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d2715-3c91-4acd-b65c-f136c73e8424 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:06.754033 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d2715-3c91-4acd-b65c-f136c73e8424 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:06.754927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04d2715-3c91-4acd-b65c-f136c73e8424 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:06.761317 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04d2715-3c91-4acd-b65c-f136c73e8424 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:06.763584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 16/31] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:06.767272 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6d9ba-161a-49a9-aade-3e2ecf23ad6b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:06.903239 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6d9ba-161a-49a9-aade-3e2ecf23ad6b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:06.940656 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff0058-1d1c-414f-a537-7a19e3ffb190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:06.941332 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff0058-1d1c-414f-a537-7a19e3ffb190 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:06.941614 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff0058-1d1c-414f-a537-7a19e3ffb190 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:06.941937 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff0058-1d1c-414f-a537-7a19e3ffb190 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:06.954260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff0058-1d1c-414f-a537-7a19e3ffb190 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:06.955847 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff0058-1d1c-414f-a537-7a19e3ffb190 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:06.957578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 16/32] 10.222.0.17 () {60 vars in 1131 bytes} [Sat Sep 10 23:41:06 2022] POST /identity/v3/roles => generated 219 bytes in 592 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:41:06.958775 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d2715-3c91-4acd-b65c-f136c73e8424 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:06.978958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-afff0058-1d1c-414f-a537-7a19e3ffb190 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:41:06.978958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:06.978958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:06.978958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:06.978958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:06.978958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:06.978958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:06.978958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:06.978958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:06.978958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:06.978958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:06.978958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:06.978958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:41:06.978958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:41:06.978958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:41:06.978958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:41:06.979887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:06.979887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:06.979887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:06.979887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:06.979887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:06.979887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:06.979887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:06.979887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:06.979887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:06.979887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:06.979887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:06.979887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:06.979887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:06.979887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:06.979887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:06.979887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:06.979887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:41:06.980667 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:41:06.980667 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:41:06.980667 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:41:06.980667 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 17/33] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:41:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:06.981295 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff0058-1d1c-414f-a537-7a19e3ffb190 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:07.085312 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:07.155593 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d2715-3c91-4acd-b65c-f136c73e8424 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:07.205683 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff0058-1d1c-414f-a537-7a19e3ffb190 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:07.214315 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7296c-9651-4620-9ce1-0601f6f92b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:07.214960 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7296c-9651-4620-9ce1-0601f6f92b9b None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:07.215235 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7296c-9651-4620-9ce1-0601f6f92b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:07.216081 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7296c-9651-4620-9ce1-0601f6f92b9b None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:07.217362 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7296c-9651-4620-9ce1-0601f6f92b9b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:07.218965 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7296c-9651-4620-9ce1-0601f6f92b9b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:07.253783 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2653ef0f-1de1-4f46-b761-72eee17c5a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:07.254417 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653ef0f-1de1-4f46-b761-72eee17c5a80 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:07.254756 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653ef0f-1de1-4f46-b761-72eee17c5a80 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:07.254990 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653ef0f-1de1-4f46-b761-72eee17c5a80 None admin] PATH_INFO: `/v3/users/5243f89b8bf44f868ec5406d662b5cd9` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:07.256133 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2653ef0f-1de1-4f46-b761-72eee17c5a80 None admin] RBAC: Authorizing `identity:get_user(user_id=5243f89b8bf44f868ec5406d662b5cd9)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:07.257597 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2653ef0f-1de1-4f46-b761-72eee17c5a80 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:07.259275 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 18/34] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:06 2022] GET /identity/v3/users/5243f89b8bf44f868ec5406d662b5cd9 => generated 247 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:07.260077 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2653ef0f-1de1-4f46-b761-72eee17c5a80 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:07.305211 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 17/35] 10.222.0.17 () {60 vars in 1131 bytes} [Sat Sep 10 23:41:06 2022] POST /identity/v3/roles => generated 225 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:41:07.306114 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7296c-9651-4620-9ce1-0601f6f92b9b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:07.376987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:07.487843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2653ef0f-1de1-4f46-b761-72eee17c5a80 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:07.527537 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:07.533703 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7296c-9651-4620-9ce1-0601f6f92b9b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:07.556980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23ee17-58b1-4f04-ac92-61900e5b6cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:07.558102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23ee17-58b1-4f04-ac92-61900e5b6cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:07.558553 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23ee17-58b1-4f04-ac92-61900e5b6cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:07.559119 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23ee17-58b1-4f04-ac92-61900e5b6cc8 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:07.559929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a23ee17-58b1-4f04-ac92-61900e5b6cc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:07.560595 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a23ee17-58b1-4f04-ac92-61900e5b6cc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:07.563415 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a23ee17-58b1-4f04-ac92-61900e5b6cc8 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:07.578257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258a723-5143-4f08-8d27-308ac0897e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:07.578928 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258a723-5143-4f08-8d27-308ac0897e01 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:07.579207 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258a723-5143-4f08-8d27-308ac0897e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:07.579476 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258a723-5143-4f08-8d27-308ac0897e01 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:07.580967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d258a723-5143-4f08-8d27-308ac0897e01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:07.582975 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d258a723-5143-4f08-8d27-308ac0897e01 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:07.584712 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 18/36] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:41:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:07.585812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 19/37] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:07.585988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1da49-5647-407f-972a-08ff0e166113 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:07.586270 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1da49-5647-407f-972a-08ff0e166113 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:07.586545 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1da49-5647-407f-972a-08ff0e166113 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:07.588958 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 19/38] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:07.591524 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23ee17-58b1-4f04-ac92-61900e5b6cc8 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:07.594069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1da49-5647-407f-972a-08ff0e166113 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:07.703330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:07.703921 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23ee17-58b1-4f04-ac92-61900e5b6cc8 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:07.710932 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1da49-5647-407f-972a-08ff0e166113 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:07.737711 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204a28f-76d8-4877-8eab-2ab669d40b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:07.738355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204a28f-76d8-4877-8eab-2ab669d40b08 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:07.738579 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204a28f-76d8-4877-8eab-2ab669d40b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:07.738875 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204a28f-76d8-4877-8eab-2ab669d40b08 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:07.739532 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3204a28f-76d8-4877-8eab-2ab669d40b08 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:07.740974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3204a28f-76d8-4877-8eab-2ab669d40b08 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:07.743704 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e1ccf-fdca-4b07-beb5-f74fab11dd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:07.744353 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e1ccf-fdca-4b07-beb5-f74fab11dd67 None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:07.744627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e1ccf-fdca-4b07-beb5-f74fab11dd67 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:07.744896 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e1ccf-fdca-4b07-beb5-f74fab11dd67 None admin] PATH_INFO: `/v3/domains/default/users/5243f89b8bf44f868ec5406d662b5cd9/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:07.747064 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e1ccf-fdca-4b07-beb5-f74fab11dd67 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=5243f89b8bf44f868ec5406d662b5cd9, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:07.749271 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e1ccf-fdca-4b07-beb5-f74fab11dd67 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:07.847204 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 20/39] 10.222.0.17 () {58 vars in 1361 bytes} [Sat Sep 10 23:41:07 2022] PUT /identity/v3/domains/default/users/5243f89b8bf44f868ec5406d662b5cd9/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:41:07.848515 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd137a-3100-4ad7-852c-4d978c606caf None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:07.848801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd137a-3100-4ad7-852c-4d978c606caf None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:07.849111 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd137a-3100-4ad7-852c-4d978c606caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:07.871145 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-fffd137a-3100-4ad7-852c-4d978c606caf None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:07.872731 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-fffd137a-3100-4ad7-852c-4d978c606caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:07.949557 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 21/40] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:07.950929 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feff42c-bdc3-4f17-9889-9a237638bd06 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:07.951288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feff42c-bdc3-4f17-9889-9a237638bd06 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:07.951613 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feff42c-bdc3-4f17-9889-9a237638bd06 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:07.953632 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 22/41] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:07.960178 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb9bf8-d0a9-4aa5-9c48-da908e4e4fcc None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:07.960460 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb9bf8-d0a9-4aa5-9c48-da908e4e4fcc None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:07.960739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb9bf8-d0a9-4aa5-9c48-da908e4e4fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:07.981611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-15eb9bf8-d0a9-4aa5-9c48-da908e4e4fcc None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:07.983763 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-15eb9bf8-d0a9-4aa5-9c48-da908e4e4fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:08.072947 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 23/42] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:08.110829 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 20/43] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Sep 10 23:41:07 2022] POST /identity/v3/projects => generated 301 bytes in 520 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 23:41:08.267243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb9bf8-d0a9-4aa5-9c48-da908e4e4fcc None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:08.283728 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:08.350245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204a28f-76d8-4877-8eab-2ab669d40b08 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:08.385100 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:08.397050 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb9bf8-d0a9-4aa5-9c48-da908e4e4fcc None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:08.436969 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ae620-22f1-4f13-a2af-a7360b481e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:08.437582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ae620-22f1-4f13-a2af-a7360b481e4d None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:08.437859 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ae620-22f1-4f13-a2af-a7360b481e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:08.438180 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ae620-22f1-4f13-a2af-a7360b481e4d None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:08.438890 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281ae620-22f1-4f13-a2af-a7360b481e4d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:08.440284 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281ae620-22f1-4f13-a2af-a7360b481e4d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:08.464683 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204a28f-76d8-4877-8eab-2ab669d40b08 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:08.498440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf0735-0dc5-46dc-ae9e-1d01f6f109ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:08.499215 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf0735-0dc5-46dc-ae9e-1d01f6f109ae None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:08.499501 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf0735-0dc5-46dc-ae9e-1d01f6f109ae None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:08.499776 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf0735-0dc5-46dc-ae9e-1d01f6f109ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:08.500966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf0735-0dc5-46dc-ae9e-1d01f6f109ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:08.502529 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf0735-0dc5-46dc-ae9e-1d01f6f109ae None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:08.504121 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 21/44] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:41:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:41:08.509448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf0735-0dc5-46dc-ae9e-1d01f6f109ae None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:08.541102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 24/45] 10.222.0.17 () {60 vars in 1131 bytes} [Sat Sep 10 23:41:08 2022] POST /identity/v3/roles => generated 223 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:41:08.616042 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:08.623974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf0735-0dc5-46dc-ae9e-1d01f6f109ae None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:08.661546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b93cea-0a1b-40bb-b0e0-43a6b5832ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:08.662163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b93cea-0a1b-40bb-b0e0-43a6b5832ce4 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:08.662443 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b93cea-0a1b-40bb-b0e0-43a6b5832ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:08.662739 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b93cea-0a1b-40bb-b0e0-43a6b5832ce4 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:08.663404 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b93cea-0a1b-40bb-b0e0-43a6b5832ce4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:08.664880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b93cea-0a1b-40bb-b0e0-43a6b5832ce4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:08.739594 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 22/46] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Sep 10 23:41:08 2022] POST /identity/v3/projects => generated 312 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 23:41:10.094113 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90d88d-73ff-43ee-81bc-7f0d2a9cbb34 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:10.094628 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90d88d-73ff-43ee-81bc-7f0d2a9cbb34 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:10.094731 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90d88d-73ff-43ee-81bc-7f0d2a9cbb34 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:10.096027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 25/47] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:41:10.101482 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda33b08-51f9-49f4-bf06-44b3adedbacc None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:10.101885 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda33b08-51f9-49f4-bf06-44b3adedbacc None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:10.102031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda33b08-51f9-49f4-bf06-44b3adedbacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:10.120705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-dda33b08-51f9-49f4-bf06-44b3adedbacc None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:10.122194 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-dda33b08-51f9-49f4-bf06-44b3adedbacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:10.186112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 23/48] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:10.458153 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90d88d-73ff-43ee-81bc-7f0d2a9cbb34 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:10.534633 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:10.559396 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90d88d-73ff-43ee-81bc-7f0d2a9cbb34 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:10.592618 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8aa7b9-7889-4ceb-96a4-283c4a568e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:10.593234 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8aa7b9-7889-4ceb-96a4-283c4a568e3b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:10.593508 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8aa7b9-7889-4ceb-96a4-283c4a568e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:10.593774 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8aa7b9-7889-4ceb-96a4-283c4a568e3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:10.605631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8aa7b9-7889-4ceb-96a4-283c4a568e3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:10.612101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-3b8aa7b9-7889-4ceb-96a4-283c4a568e3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:41:10.612101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:10.612101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:10.612101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:10.612101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:10.612101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:10.612101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:10.612101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:10.612101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:10.612101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:10.612101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:10.612101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:10.612101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:10.612101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:10.612101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:10.612101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:10.612976 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:10.612976 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:10.612976 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:10.612976 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:10.612976 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:10.612976 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:10.612976 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:10.612976 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:10.612976 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:10.612976 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:10.612976 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:10.612976 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:10.612976 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:10.612976 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:10.612976 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:10.612976 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:10.613725 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:10.613725 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:10.613725 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:10.613725 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:10.613725 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:10.613725 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:10.613725 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:10.613725 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:10.613725 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:10.613725 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:10.613725 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:41:10.613725 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:41:10.613725 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 26/49] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:41:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:10.618989 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda33b08-51f9-49f4-bf06-44b3adedbacc None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:10.693250 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:10.713907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda33b08-51f9-49f4-bf06-44b3adedbacc None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:10.746906 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe230db-fcfd-46b3-a188-3ffb2a48ff7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:10.747501 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe230db-fcfd-46b3-a188-3ffb2a48ff7e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:10.747780 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe230db-fcfd-46b3-a188-3ffb2a48ff7e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:10.748580 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe230db-fcfd-46b3-a188-3ffb2a48ff7e None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:10.749241 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe230db-fcfd-46b3-a188-3ffb2a48ff7e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:10.749535 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe230db-fcfd-46b3-a188-3ffb2a48ff7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:10.750960 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe230db-fcfd-46b3-a188-3ffb2a48ff7e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:10.760154 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 24/50] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:10.766055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8aa7b9-7889-4ceb-96a4-283c4a568e3b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:10.868435 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8aa7b9-7889-4ceb-96a4-283c4a568e3b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:10.901056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e418cc-5b3e-416f-b251-7a8cafdd7519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:10.901676 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e418cc-5b3e-416f-b251-7a8cafdd7519 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:10.901950 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e418cc-5b3e-416f-b251-7a8cafdd7519 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:10.902214 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e418cc-5b3e-416f-b251-7a8cafdd7519 None admin] PATH_INFO: `/v3/users/5243f89b8bf44f868ec5406d662b5cd9` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:10.903349 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e418cc-5b3e-416f-b251-7a8cafdd7519 None admin] RBAC: Authorizing `identity:get_user(user_id=5243f89b8bf44f868ec5406d662b5cd9)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:10.904824 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e418cc-5b3e-416f-b251-7a8cafdd7519 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:10.906380 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 27/51] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:10 2022] GET /identity/v3/users/5243f89b8bf44f868ec5406d662b5cd9 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:10.911955 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe230db-fcfd-46b3-a188-3ffb2a48ff7e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:10.936177 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03168572-0310-4202-acd3-36de1944e2d9 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:10.936177 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03168572-0310-4202-acd3-36de1944e2d9 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:10.936781 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03168572-0310-4202-acd3-36de1944e2d9 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:10.937974 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 28/52] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:41:10.943109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d3f4f-bd0c-4f51-9d46-62cc43d1f30b None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:10.943501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d3f4f-bd0c-4f51-9d46-62cc43d1f30b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:10.943775 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d3f4f-bd0c-4f51-9d46-62cc43d1f30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:10.949991 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:10.962181 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-043d3f4f-bd0c-4f51-9d46-62cc43d1f30b None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:10.963399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-043d3f4f-bd0c-4f51-9d46-62cc43d1f30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:11.012222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe230db-fcfd-46b3-a188-3ffb2a48ff7e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:11.035039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 29/53] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 23:41:11.048297 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac26d5-6929-473b-bad8-41d18ed7d344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:11.053366 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac26d5-6929-473b-bad8-41d18ed7d344 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:11.053366 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac26d5-6929-473b-bad8-41d18ed7d344 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:11.053366 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac26d5-6929-473b-bad8-41d18ed7d344 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:11.053366 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac26d5-6929-473b-bad8-41d18ed7d344 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:11.053366 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac26d5-6929-473b-bad8-41d18ed7d344 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:11.053875 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 25/54] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:41:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:11.059587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d3f4f-bd0c-4f51-9d46-62cc43d1f30b None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:11.266263 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d3f4f-bd0c-4f51-9d46-62cc43d1f30b None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:11.298609 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:11.303739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6779bbe-a6ba-4640-99cc-4f9f8bd1abd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:11.304554 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6779bbe-a6ba-4640-99cc-4f9f8bd1abd9 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:11.305390 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6779bbe-a6ba-4640-99cc-4f9f8bd1abd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:11.306206 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6779bbe-a6ba-4640-99cc-4f9f8bd1abd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:11.306726 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac26d5-6929-473b-bad8-41d18ed7d344 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:11.307096 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6779bbe-a6ba-4640-99cc-4f9f8bd1abd9 None admin] RBAC: Adding query filter params (role.id=9dce48a2b53c482d931dcd064b07744e, scope.domain.id=default, user.id=5243f89b8bf44f868ec5406d662b5cd9) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:11.311289 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6779bbe-a6ba-4640-99cc-4f9f8bd1abd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:11.311289 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6779bbe-a6ba-4640-99cc-4f9f8bd1abd9 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:11.317590 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 30/55] 10.222.0.17 () {56 vars in 1322 bytes} [Sat Sep 10 23:41:11 2022] GET /identity/v3/role_assignments?user.id=5243f89b8bf44f868ec5406d662b5cd9&scope.domain.id=default&role.id=9dce48a2b53c482d931dcd064b07744e => generated 530 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:11.320732 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:11.416416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac26d5-6929-473b-bad8-41d18ed7d344 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:11.453953 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71847020-a40b-4aa7-9fd6-843649df6727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:11.454584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71847020-a40b-4aa7-9fd6-843649df6727 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:11.454892 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71847020-a40b-4aa7-9fd6-843649df6727 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:11.455163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71847020-a40b-4aa7-9fd6-843649df6727 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:11.456407 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71847020-a40b-4aa7-9fd6-843649df6727 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:11.457870 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71847020-a40b-4aa7-9fd6-843649df6727 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:11.459534 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 26/56] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:41:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:41:11.465456 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6779bbe-a6ba-4640-99cc-4f9f8bd1abd9 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:11.577677 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6779bbe-a6ba-4640-99cc-4f9f8bd1abd9 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:11.610859 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf296a-be39-4306-ab87-c38b04280932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:11.611463 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf296a-be39-4306-ab87-c38b04280932 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:11.611739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf296a-be39-4306-ab87-c38b04280932 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:11.612006 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf296a-be39-4306-ab87-c38b04280932 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:11.612647 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf296a-be39-4306-ab87-c38b04280932 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:11.614088 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf296a-be39-4306-ab87-c38b04280932 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:11.686781 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 31/57] 10.222.0.17 () {60 vars in 1140 bytes} [Sat Sep 10 23:41:11 2022] POST /identity/v3/projects => generated 298 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:41:13.805709 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4258361-1742-4aff-baef-16bbff6cb47c None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:13.806251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4258361-1742-4aff-baef-16bbff6cb47c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:13.806251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4258361-1742-4aff-baef-16bbff6cb47c None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:13.807879 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 27/58] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:13.813778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c3827-8bb5-4190-ac36-1b920932dc99 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:13.814175 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c3827-8bb5-4190-ac36-1b920932dc99 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:13.814332 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c3827-8bb5-4190-ac36-1b920932dc99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:13.833011 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-3b2c3827-8bb5-4190-ac36-1b920932dc99 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:13.834204 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3b2c3827-8bb5-4190-ac36-1b920932dc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:13.877864 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:13.899942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 32/59] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:14.176691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4258361-1742-4aff-baef-16bbff6cb47c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:14.255029 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:14.277448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4258361-1742-4aff-baef-16bbff6cb47c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:14.310742 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133b3b6-9bb9-463a-8417-6784289aa08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:14.311446 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133b3b6-9bb9-463a-8417-6784289aa08b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:14.311718 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133b3b6-9bb9-463a-8417-6784289aa08b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:14.311989 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133b3b6-9bb9-463a-8417-6784289aa08b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:14.313077 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6133b3b6-9bb9-463a-8417-6784289aa08b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:14.314531 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6133b3b6-9bb9-463a-8417-6784289aa08b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:14.316118 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 28/60] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:41:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:14.321546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c3827-8bb5-4190-ac36-1b920932dc99 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:14.417496 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c3827-8bb5-4190-ac36-1b920932dc99 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:14.450166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5fb64-114a-4bd3-8154-696a3c1f58f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:14.450722 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5fb64-114a-4bd3-8154-696a3c1f58f9 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:14.451023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5fb64-114a-4bd3-8154-696a3c1f58f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:14.451301 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5fb64-114a-4bd3-8154-696a3c1f58f9 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:14.451988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef5fb64-114a-4bd3-8154-696a3c1f58f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:14.453438 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef5fb64-114a-4bd3-8154-696a3c1f58f9 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:14.539835 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 33/61] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Sep 10 23:41:14 2022] POST /identity/v3/users => generated 275 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:41:17.882883 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af435e1-caff-4076-8118-1042b7b2d70f None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:17.883425 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af435e1-caff-4076-8118-1042b7b2d70f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:17.883425 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af435e1-caff-4076-8118-1042b7b2d70f None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:17.884787 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 29/62] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:17.890702 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95dc459-ec79-4778-8130-b070afb46d3e None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:17.891109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95dc459-ec79-4778-8130-b070afb46d3e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:17.891271 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95dc459-ec79-4778-8130-b070afb46d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:17.920449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-c95dc459-ec79-4778-8130-b070afb46d3e None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:17.921667 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c95dc459-ec79-4778-8130-b070afb46d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:17.926496 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf3271-98e4-4b70-8eb0-15999382d38c None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:17.926810 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf3271-98e4-4b70-8eb0-15999382d38c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:17.927089 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf3271-98e4-4b70-8eb0-15999382d38c None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:17.928693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 30/63] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:17.943569 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcbcc04-a43b-4b41-b311-4bd38773c880 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:17.944151 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcbcc04-a43b-4b41-b311-4bd38773c880 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:17.944627 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcbcc04-a43b-4b41-b311-4bd38773c880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:17.955275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:17.978904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-fdcbcc04-a43b-4b41-b311-4bd38773c880 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:17.980121 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fdcbcc04-a43b-4b41-b311-4bd38773c880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:17.998781 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 34/64] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:18.091616 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 31/65] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 23:41:18.189470 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe731e94-be4a-405b-8b73-6c36e9d30f93 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:18.190097 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe731e94-be4a-405b-8b73-6c36e9d30f93 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:18.190559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe731e94-be4a-405b-8b73-6c36e9d30f93 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:18.192106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 35/66] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:18.197581 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c4b47-fff3-4917-abb7-37f1ed080dac None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:18.198109 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c4b47-fff3-4917-abb7-37f1ed080dac None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:18.198573 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c4b47-fff3-4917-abb7-37f1ed080dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:18.223896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-2d4c4b47-fff3-4917-abb7-37f1ed080dac None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:18.225117 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2d4c4b47-fff3-4917-abb7-37f1ed080dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:18.321564 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 32/67] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:18.409155 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe731e94-be4a-405b-8b73-6c36e9d30f93 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:18.515514 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c4b47-fff3-4917-abb7-37f1ed080dac None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:18.523542 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:18.536696 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe731e94-be4a-405b-8b73-6c36e9d30f93 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:18.574330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28bcd8e-fb9a-45c2-8806-97c0a6122b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:18.574984 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bcd8e-fb9a-45c2-8806-97c0a6122b3e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:18.575261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bcd8e-fb9a-45c2-8806-97c0a6122b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:18.575527 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bcd8e-fb9a-45c2-8806-97c0a6122b3e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:18.586291 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28bcd8e-fb9a-45c2-8806-97c0a6122b3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:18.591069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-f28bcd8e-fb9a-45c2-8806-97c0a6122b3e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:18.591069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:18.591069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:18.591069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:18.591069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:18.591069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:18.591069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:18.591069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:18.591069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:18.591069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:18.591069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:18.591069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:18.591069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:18.591069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:18.591069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:18.591069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:18.592688 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:18.592688 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:18.592688 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:18.592688 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:18.592688 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:18.592688 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:18.592688 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:18.592688 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:18.592688 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:18.592688 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:18.592688 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:18.592688 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:18.592688 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:18.592688 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:18.592688 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:18.592688 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:18.593481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:18.593481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:18.593481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:18.593481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:18.593481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:18.593481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:18.593481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:18.593481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:18.593481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:18.593481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:18.593481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:18.593481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:41:18.593481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 36/68] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:18.594642 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd75bf5-b487-4c6c-b338-be6002cb76ed None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:18.594642 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd75bf5-b487-4c6c-b338-be6002cb76ed None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:18.594964 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd75bf5-b487-4c6c-b338-be6002cb76ed None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:18.596344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 37/69] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:18.597580 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5a2a9-378e-4d7b-a5b7-2b09859dfb6e None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:18.597580 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5a2a9-378e-4d7b-a5b7-2b09859dfb6e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:18.597857 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5a2a9-378e-4d7b-a5b7-2b09859dfb6e None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:18.599176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 38/70] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:18.599871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5a2a9-378e-4d7b-a5b7-2b09859dfb6e None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:18.628024 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:18.647804 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c4b47-fff3-4917-abb7-37f1ed080dac None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:18.682334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f9e28-e49e-46fd-9df0-a5ce4d22def9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:18.683010 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f9e28-e49e-46fd-9df0-a5ce4d22def9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:18.683296 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f9e28-e49e-46fd-9df0-a5ce4d22def9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:18.683575 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f9e28-e49e-46fd-9df0-a5ce4d22def9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:18.694416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f9e28-e49e-46fd-9df0-a5ce4d22def9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:18.699334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-6b3f9e28-e49e-46fd-9df0-a5ce4d22def9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:41:18.699334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:18.699334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:18.699334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:18.699334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:18.699334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:18.699334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:18.699334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:18.699334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:18.699334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:18.699334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:18.699334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:18.699334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:18.699334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:18.699334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:18.699334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:18.700164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:18.700164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:18.700164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:18.700164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:18.700164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:18.700164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:18.700164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:18.700164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:18.700164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:18.700164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:18.700164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:18.700164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:18.700164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:18.700164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:18.700164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:18.700164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:18.701453 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:18.701453 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:18.701453 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:18.701453 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:18.701453 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:18.701453 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:18.701453 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:18.701453 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:18.701453 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:18.701453 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:18.701453 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:41:18.701453 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:41:18.701453 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 33/71] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:41:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:18.703980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e5e77-30d9-441e-b4b9-95f061be7a8b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:18.704266 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e5e77-30d9-441e-b4b9-95f061be7a8b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:18.704533 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e5e77-30d9-441e-b4b9-95f061be7a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:18.709129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5a2a9-378e-4d7b-a5b7-2b09859dfb6e None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:18.721726 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-061e5e77-30d9-441e-b4b9-95f061be7a8b None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:18.722955 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-061e5e77-30d9-441e-b4b9-95f061be7a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:18.742926 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e371ac-4913-4e3f-9187-1f08d9d66379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:18.743532 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e371ac-4913-4e3f-9187-1f08d9d66379 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:18.743811 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e371ac-4913-4e3f-9187-1f08d9d66379 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:18.744084 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e371ac-4913-4e3f-9187-1f08d9d66379 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:18.744748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e371ac-4913-4e3f-9187-1f08d9d66379 None admin] RBAC: Adding query filter params (name=member) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:18.745050 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e371ac-4913-4e3f-9187-1f08d9d66379 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:18.746455 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e371ac-4913-4e3f-9187-1f08d9d66379 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:18.758860 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 39/72] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:41:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:18.760029 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd157c18-deda-447d-865c-331eee9d8f22 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:18.760342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd157c18-deda-447d-865c-331eee9d8f22 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:18.760585 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd157c18-deda-447d-865c-331eee9d8f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:18.777867 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-bd157c18-deda-447d-865c-331eee9d8f22 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:18.779667 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-bd157c18-deda-447d-865c-331eee9d8f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:18.781098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:18.789273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 34/73] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:18.790097 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e5e77-30d9-441e-b4b9-95f061be7a8b None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:18.851757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 40/74] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:18.852623 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd157c18-deda-447d-865c-331eee9d8f22 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:18.879571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:18.902166 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e5e77-30d9-441e-b4b9-95f061be7a8b None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:18.938095 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06bf65-af1d-4fe7-a58e-7e29912e8dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:18.938741 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06bf65-af1d-4fe7-a58e-7e29912e8dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:18.939026 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06bf65-af1d-4fe7-a58e-7e29912e8dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:18.939292 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06bf65-af1d-4fe7-a58e-7e29912e8dd0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:18.949773 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06bf65-af1d-4fe7-a58e-7e29912e8dd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:18.954985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-7a06bf65-af1d-4fe7-a58e-7e29912e8dd0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 23:41:18.954985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:18.954985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:18.954985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:18.954985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:18.954985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:18.954985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:18.954985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:18.954985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:18.954985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:18.954985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:18.954985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:18.954985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:18.954985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:18.954985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:18.954985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:18.955834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:18.955834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:18.955834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:18.955834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:18.955834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:18.955834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:18.955834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:18.955834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:18.955834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:18.955834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:18.955834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:18.955834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:18.955834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:18.955834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:18.955834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:18.955834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:18.956583 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:18.956583 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:18.956583 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:18.956583 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:18.956583 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:18.956583 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:18.956583 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:18.956583 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:18.956583 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:18.956583 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:18.956583 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 23:41:18.956583 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:41:18.956583 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 35/75] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Sep 10 23:41:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:18.957260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06bf65-af1d-4fe7-a58e-7e29912e8dd0 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:18.979286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd157c18-deda-447d-865c-331eee9d8f22 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:19.015468 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:19.022813 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a97ed7-20bc-4140-b24f-de9863d4c40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:19.023946 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a97ed7-20bc-4140-b24f-de9863d4c40e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:19.024425 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a97ed7-20bc-4140-b24f-de9863d4c40e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:19.024900 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a97ed7-20bc-4140-b24f-de9863d4c40e None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:19.025852 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a97ed7-20bc-4140-b24f-de9863d4c40e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:19.026352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a97ed7-20bc-4140-b24f-de9863d4c40e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:19.029550 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a97ed7-20bc-4140-b24f-de9863d4c40e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:19.041234 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 41/76] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:19.042360 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a97ed7-20bc-4140-b24f-de9863d4c40e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:19.062493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06bf65-af1d-4fe7-a58e-7e29912e8dd0 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:19.102263 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62081081-32e6-4a10-abfe-529928fb3b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:19.103252 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62081081-32e6-4a10-abfe-529928fb3b89 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:19.103713 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62081081-32e6-4a10-abfe-529928fb3b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:19.104163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62081081-32e6-4a10-abfe-529928fb3b89 None admin] PATH_INFO: `/v3/users/067af36c9f9b401d9d35810237e9ba2e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:19.122481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62081081-32e6-4a10-abfe-529928fb3b89 None admin] RBAC: Authorizing `identity:get_user(user_id=067af36c9f9b401d9d35810237e9ba2e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:19.124288 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62081081-32e6-4a10-abfe-529928fb3b89 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:19.126042 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 36/77] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:18 2022] GET /identity/v3/users/067af36c9f9b401d9d35810237e9ba2e => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:19.126911 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62081081-32e6-4a10-abfe-529928fb3b89 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:19.144926 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:19.166078 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a97ed7-20bc-4140-b24f-de9863d4c40e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:19.200300 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a0c7a8-9a12-489a-a1f7-cdbe660f064b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:19.200934 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0c7a8-9a12-489a-a1f7-cdbe660f064b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:19.201215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0c7a8-9a12-489a-a1f7-cdbe660f064b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:19.201560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0c7a8-9a12-489a-a1f7-cdbe660f064b None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:19.202245 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a0c7a8-9a12-489a-a1f7-cdbe660f064b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:19.202542 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a0c7a8-9a12-489a-a1f7-cdbe660f064b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:19.203946 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a0c7a8-9a12-489a-a1f7-cdbe660f064b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:19.211675 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 42/78] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:41:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:19.212458 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a0c7a8-9a12-489a-a1f7-cdbe660f064b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:19.225925 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62081081-32e6-4a10-abfe-529928fb3b89 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:19.239585 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:19.259846 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996a215-b613-4dab-9564-56c72342c4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:19.260464 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996a215-b613-4dab-9564-56c72342c4cf None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:19.260743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996a215-b613-4dab-9564-56c72342c4cf None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:19.261014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996a215-b613-4dab-9564-56c72342c4cf None admin] PATH_INFO: `/v3/users/5243f89b8bf44f868ec5406d662b5cd9` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:19.262171 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6996a215-b613-4dab-9564-56c72342c4cf None admin] RBAC: Authorizing `identity:get_user(user_id=5243f89b8bf44f868ec5406d662b5cd9)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:19.263860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6996a215-b613-4dab-9564-56c72342c4cf None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:19.265488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 37/79] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:19 2022] GET /identity/v3/users/5243f89b8bf44f868ec5406d662b5cd9 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:19.266214 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996a215-b613-4dab-9564-56c72342c4cf None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:19.322211 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a0c7a8-9a12-489a-a1f7-cdbe660f064b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:19.355053 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3774c-1ecd-46b2-87f5-efdcbf1f4d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:19.355673 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3774c-1ecd-46b2-87f5-efdcbf1f4d7f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:19.355953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3774c-1ecd-46b2-87f5-efdcbf1f4d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:19.356228 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3774c-1ecd-46b2-87f5-efdcbf1f4d7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:19.358443 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c3774c-1ecd-46b2-87f5-efdcbf1f4d7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:19.359912 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c3774c-1ecd-46b2-87f5-efdcbf1f4d7f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:19.361474 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 43/80] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:41:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:19.362234 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3774c-1ecd-46b2-87f5-efdcbf1f4d7f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:19.368591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996a215-b613-4dab-9564-56c72342c4cf None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:19.373396 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:19.402428 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0935c0-9b9f-4819-be07-4cd48c6055fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:19.403142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0935c0-9b9f-4819-be07-4cd48c6055fd None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:19.403423 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0935c0-9b9f-4819-be07-4cd48c6055fd None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:19.403698 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0935c0-9b9f-4819-be07-4cd48c6055fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:19.455915 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:19.464170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3774c-1ecd-46b2-87f5-efdcbf1f4d7f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:19.474611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0935c0-9b9f-4819-be07-4cd48c6055fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:19.476149 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0935c0-9b9f-4819-be07-4cd48c6055fd None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:19.497608 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e4a55-16f8-4dd2-9591-9c6d77152b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:19.498812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e4a55-16f8-4dd2-9591-9c6d77152b0e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:19.499094 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e4a55-16f8-4dd2-9591-9c6d77152b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:19.499367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e4a55-16f8-4dd2-9591-9c6d77152b0e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:19.509411 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451e4a55-16f8-4dd2-9591-9c6d77152b0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:19.514767 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-451e4a55-16f8-4dd2-9591-9c6d77152b0e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:19.514767 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:19.514767 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:19.514767 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:19.514767 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:19.514767 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:19.514767 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:19.514767 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:19.514767 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:19.514767 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:19.514767 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:19.514767 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:19.514767 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:19.514767 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:19.514767 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:19.514767 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:19.515848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:19.515848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:19.515848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:19.515848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:19.515848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:19.515848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:19.515848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:19.515848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:19.515848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:19.515848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:19.515848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:19.515848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:19.515848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:19.515848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:19.515848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:19.515848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:19.520030 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:19.520030 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:19.520030 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:19.520030 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:19.520030 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:19.520030 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:19.520030 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:19.520030 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:19.520030 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:19.520030 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:19.520030 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:19.520030 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:41:19.520030 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 44/81] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:19.520030 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e4a55-16f8-4dd2-9591-9c6d77152b0e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:19.582028 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 38/82] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:41:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:41:19.582830 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0935c0-9b9f-4819-be07-4cd48c6055fd None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:19.587186 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:19.616623 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e4a55-16f8-4dd2-9591-9c6d77152b0e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:19.650363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714510ae-6f3d-493e-a93c-ddd6a52f9127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:19.651019 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714510ae-6f3d-493e-a93c-ddd6a52f9127 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:19.651301 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714510ae-6f3d-493e-a93c-ddd6a52f9127 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:19.651573 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714510ae-6f3d-493e-a93c-ddd6a52f9127 None admin] PATH_INFO: `/v3/users/067af36c9f9b401d9d35810237e9ba2e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:19.652694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714510ae-6f3d-493e-a93c-ddd6a52f9127 None admin] RBAC: Authorizing `identity:get_user(user_id=067af36c9f9b401d9d35810237e9ba2e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:19.654650 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714510ae-6f3d-493e-a93c-ddd6a52f9127 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:19.656634 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 45/83] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:19 2022] GET /identity/v3/users/067af36c9f9b401d9d35810237e9ba2e => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:19.657377 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714510ae-6f3d-493e-a93c-ddd6a52f9127 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:19.661675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:19.680553 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0935c0-9b9f-4819-be07-4cd48c6055fd None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:19.714457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076ac97-fef5-4faf-8a4c-38f795e8c96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:19.715084 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076ac97-fef5-4faf-8a4c-38f795e8c96a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:19.715366 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076ac97-fef5-4faf-8a4c-38f795e8c96a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:19.715641 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076ac97-fef5-4faf-8a4c-38f795e8c96a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:19.761360 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714510ae-6f3d-493e-a93c-ddd6a52f9127 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:19.783407 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c076ac97-fef5-4faf-8a4c-38f795e8c96a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:19.784876 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c076ac97-fef5-4faf-8a4c-38f795e8c96a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:19.795761 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4bdf4-5718-4c77-9d57-b747dd2e0ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:19.796368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4bdf4-5718-4c77-9d57-b747dd2e0ff2 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:19.796651 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4bdf4-5718-4c77-9d57-b747dd2e0ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:19.796922 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4bdf4-5718-4c77-9d57-b747dd2e0ff2 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:19.797567 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf4bdf4-5718-4c77-9d57-b747dd2e0ff2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:19.799060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf4bdf4-5718-4c77-9d57-b747dd2e0ff2 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:19.808068 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:19.863290 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:19.887319 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 39/84] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:41:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:41:19.887319 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076ac97-fef5-4faf-8a4c-38f795e8c96a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:19.887557 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 46/85] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Sep 10 23:41:19 2022] POST /identity/v3/users => generated 289 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:41:19.887557 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4bdf4-5718-4c77-9d57-b747dd2e0ff2 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:20.000466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076ac97-fef5-4faf-8a4c-38f795e8c96a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:20.009151 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4bdf4-5718-4c77-9d57-b747dd2e0ff2 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:20.047896 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a07257-ba0e-4648-a3d2-32c3ad84dfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:20.048645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a07257-ba0e-4648-a3d2-32c3ad84dfc8 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:20.049029 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a07257-ba0e-4648-a3d2-32c3ad84dfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:20.049300 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a07257-ba0e-4648-a3d2-32c3ad84dfc8 None admin] PATH_INFO: `/v3/projects/59ae2524fa884ab7a9682a93d368b500` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:20.050443 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a07257-ba0e-4648-a3d2-32c3ad84dfc8 None admin] RBAC: Authorizing `identity:get_project(project_id=59ae2524fa884ab7a9682a93d368b500)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:20.052066 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a07257-ba0e-4648-a3d2-32c3ad84dfc8 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:20.059400 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 47/86] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:19 2022] GET /identity/v3/projects/59ae2524fa884ab7a9682a93d368b500 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:20.059400 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a07257-ba0e-4648-a3d2-32c3ad84dfc8 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:20.061892 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727f5c7-6d40-49a4-9bbb-7992ea7c5095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:20.062513 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727f5c7-6d40-49a4-9bbb-7992ea7c5095 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:20.063304 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727f5c7-6d40-49a4-9bbb-7992ea7c5095 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:20.063304 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727f5c7-6d40-49a4-9bbb-7992ea7c5095 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:20.063760 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c727f5c7-6d40-49a4-9bbb-7992ea7c5095 None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:20.064065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c727f5c7-6d40-49a4-9bbb-7992ea7c5095 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:20.065457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c727f5c7-6d40-49a4-9bbb-7992ea7c5095 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:20.075094 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 40/87] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:41:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:20.075714 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727f5c7-6d40-49a4-9bbb-7992ea7c5095 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:20.126856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:20.130328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:20.175248 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a07257-ba0e-4648-a3d2-32c3ad84dfc8 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:20.203359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727f5c7-6d40-49a4-9bbb-7992ea7c5095 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:20.218324 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ce02eb-dd81-4a2d-9817-a3dcbaacf247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:20.219308 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce02eb-dd81-4a2d-9817-a3dcbaacf247 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:20.219614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce02eb-dd81-4a2d-9817-a3dcbaacf247 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:20.219856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce02eb-dd81-4a2d-9817-a3dcbaacf247 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:20.245421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c92cda-aca3-49db-aa3b-5fdf9798b7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:20.246054 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c92cda-aca3-49db-aa3b-5fdf9798b7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:20.246333 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c92cda-aca3-49db-aa3b-5fdf9798b7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:20.246605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c92cda-aca3-49db-aa3b-5fdf9798b7e0 None admin] PATH_INFO: `/v3/projects/59ae2524fa884ab7a9682a93d368b500` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:20.248634 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c92cda-aca3-49db-aa3b-5fdf9798b7e0 None admin] RBAC: Authorizing `identity:get_project(project_id=59ae2524fa884ab7a9682a93d368b500)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:20.250286 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c92cda-aca3-49db-aa3b-5fdf9798b7e0 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:20.251973 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 41/88] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:20 2022] GET /identity/v3/projects/59ae2524fa884ab7a9682a93d368b500 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:20.252755 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c92cda-aca3-49db-aa3b-5fdf9798b7e0 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:20.320050 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ce02eb-dd81-4a2d-9817-a3dcbaacf247 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:20.321655 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ce02eb-dd81-4a2d-9817-a3dcbaacf247 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:20.368085 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c92cda-aca3-49db-aa3b-5fdf9798b7e0 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:20.383277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:20.390562 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:20.419968 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a46f497-571c-42cb-a0a6-2f8b9512a226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:20.421068 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a46f497-571c-42cb-a0a6-2f8b9512a226 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:20.421648 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a46f497-571c-42cb-a0a6-2f8b9512a226 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:20.422385 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a46f497-571c-42cb-a0a6-2f8b9512a226 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:20.423316 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a46f497-571c-42cb-a0a6-2f8b9512a226 None admin] RBAC: Adding query filter params (role.id=824f5fe1f02b447382190a8c236c27db, scope.project.id=59ae2524fa884ab7a9682a93d368b500, user.id=067af36c9f9b401d9d35810237e9ba2e) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:20.423813 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a46f497-571c-42cb-a0a6-2f8b9512a226 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:20.426108 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a46f497-571c-42cb-a0a6-2f8b9512a226 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:20.434428 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 42/89] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:41:20 2022] GET /identity/v3/role_assignments?user.id=067af36c9f9b401d9d35810237e9ba2e&scope.project.id=59ae2524fa884ab7a9682a93d368b500&role.id=824f5fe1f02b447382190a8c236c27db => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:20.436657 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a46f497-571c-42cb-a0a6-2f8b9512a226 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:20.439388 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 48/90] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:41:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:41:20.443456 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ce02eb-dd81-4a2d-9817-a3dcbaacf247 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:20.563615 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a46f497-571c-42cb-a0a6-2f8b9512a226 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:20.627314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ce02eb-dd81-4a2d-9817-a3dcbaacf247 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:20.637730 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8030cb4-f14c-4a1d-bdd6-90c6f1fa556b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:20.639173 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8030cb4-f14c-4a1d-bdd6-90c6f1fa556b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:20.639612 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8030cb4-f14c-4a1d-bdd6-90c6f1fa556b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:20.640016 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8030cb4-f14c-4a1d-bdd6-90c6f1fa556b None admin] PATH_INFO: `/v3/users/067af36c9f9b401d9d35810237e9ba2e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:20.642002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8030cb4-f14c-4a1d-bdd6-90c6f1fa556b None admin] RBAC: Authorizing `identity:get_user(user_id=067af36c9f9b401d9d35810237e9ba2e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:20.643680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8030cb4-f14c-4a1d-bdd6-90c6f1fa556b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:20.648145 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 43/91] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:20 2022] GET /identity/v3/users/067af36c9f9b401d9d35810237e9ba2e => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:20.649200 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8030cb4-f14c-4a1d-bdd6-90c6f1fa556b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:20.700766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992bbc7d-2671-4146-99a4-25506fd896c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:20.702151 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bbc7d-2671-4146-99a4-25506fd896c8 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:20.702746 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bbc7d-2671-4146-99a4-25506fd896c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:20.704281 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bbc7d-2671-4146-99a4-25506fd896c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:20.705180 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992bbc7d-2671-4146-99a4-25506fd896c8 None admin] RBAC: Adding query filter params (role.id=9dce48a2b53c482d931dcd064b07744e, scope.project.id=59ae2524fa884ab7a9682a93d368b500, user.id=5243f89b8bf44f868ec5406d662b5cd9) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:20.706620 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992bbc7d-2671-4146-99a4-25506fd896c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:20.711133 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992bbc7d-2671-4146-99a4-25506fd896c8 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:20.721917 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 49/92] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:41:20 2022] GET /identity/v3/role_assignments?user.id=5243f89b8bf44f868ec5406d662b5cd9&scope.project.id=59ae2524fa884ab7a9682a93d368b500&role.id=9dce48a2b53c482d931dcd064b07744e => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:20.724783 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992bbc7d-2671-4146-99a4-25506fd896c8 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:20.727118 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:20.782460 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:20.851700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8030cb4-f14c-4a1d-bdd6-90c6f1fa556b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:20.923569 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a6b72-5cca-4277-ba76-6d9a0684f78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:20.924488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a6b72-5cca-4277-ba76-6d9a0684f78d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:20.925943 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a6b72-5cca-4277-ba76-6d9a0684f78d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:20.927036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a6b72-5cca-4277-ba76-6d9a0684f78d None admin] PATH_INFO: `/v3/projects/59ae2524fa884ab7a9682a93d368b500` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:20.928893 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a6b72-5cca-4277-ba76-6d9a0684f78d None admin] RBAC: Authorizing `identity:get_project(project_id=59ae2524fa884ab7a9682a93d368b500)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:20.930718 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a6b72-5cca-4277-ba76-6d9a0684f78d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:20.933021 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992bbc7d-2671-4146-99a4-25506fd896c8 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:20.936995 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 44/93] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:20 2022] GET /identity/v3/projects/59ae2524fa884ab7a9682a93d368b500 => generated 298 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:20.943834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a6b72-5cca-4277-ba76-6d9a0684f78d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:21.014088 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df4b42-018e-4736-9234-13215d7e0342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:21.015319 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df4b42-018e-4736-9234-13215d7e0342 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:21.015764 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df4b42-018e-4736-9234-13215d7e0342 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:21.016173 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df4b42-018e-4736-9234-13215d7e0342 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:21.112245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a6b72-5cca-4277-ba76-6d9a0684f78d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:21.133437 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:21.147397 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df4b42-018e-4736-9234-13215d7e0342 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:21.148922 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df4b42-018e-4736-9234-13215d7e0342 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:21.182211 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d14105-8752-4481-aef2-94f391f35cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:21.182765 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d14105-8752-4481-aef2-94f391f35cbc None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:21.183205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d14105-8752-4481-aef2-94f391f35cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:21.183527 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d14105-8752-4481-aef2-94f391f35cbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:21.183852 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:21.184261 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d14105-8752-4481-aef2-94f391f35cbc None admin] RBAC: Adding query filter params (role.id=ff2ed192587843e5b10b5a35949e4c5f, scope.project.id=59ae2524fa884ab7a9682a93d368b500, user.id=067af36c9f9b401d9d35810237e9ba2e) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:21.184986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d14105-8752-4481-aef2-94f391f35cbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:21.187622 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d14105-8752-4481-aef2-94f391f35cbc None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:21.202619 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 45/94] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:41:20 2022] GET /identity/v3/role_assignments?user.id=067af36c9f9b401d9d35810237e9ba2e&scope.project.id=59ae2524fa884ab7a9682a93d368b500&role.id=ff2ed192587843e5b10b5a35949e4c5f => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:21.280001 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 50/95] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:41:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:41:21.286406 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d14105-8752-4481-aef2-94f391f35cbc None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:21.399069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d14105-8752-4481-aef2-94f391f35cbc None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:21.436471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8b328-b54c-4917-9a6a-53e95baf5106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:21.437221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8b328-b54c-4917-9a6a-53e95baf5106 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:21.437498 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8b328-b54c-4917-9a6a-53e95baf5106 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:21.437764 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8b328-b54c-4917-9a6a-53e95baf5106 None admin] PATH_INFO: `/v3/projects/890c323b0dd14e24b001370bdb56e28d` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:21.440417 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8b328-b54c-4917-9a6a-53e95baf5106 None admin] RBAC: Authorizing `identity:get_project(project_id=890c323b0dd14e24b001370bdb56e28d)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:21.440647 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8b328-b54c-4917-9a6a-53e95baf5106 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:21.442462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 46/96] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:21 2022] GET /identity/v3/projects/890c323b0dd14e24b001370bdb56e28d => generated 312 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:21.448368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df4b42-018e-4736-9234-13215d7e0342 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:21.569041 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df4b42-018e-4736-9234-13215d7e0342 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:21.593950 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:21.611322 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03618b78-d65a-411c-9616-095dbb5c2261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:21.612018 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03618b78-d65a-411c-9616-095dbb5c2261 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:21.621283 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03618b78-d65a-411c-9616-095dbb5c2261 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:21.621283 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03618b78-d65a-411c-9616-095dbb5c2261 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:21.621283 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03618b78-d65a-411c-9616-095dbb5c2261 None admin] RBAC: Adding query filter params (role.id=824f5fe1f02b447382190a8c236c27db, scope.project.id=890c323b0dd14e24b001370bdb56e28d, user.id=067af36c9f9b401d9d35810237e9ba2e) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:21.621283 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03618b78-d65a-411c-9616-095dbb5c2261 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:21.621283 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03618b78-d65a-411c-9616-095dbb5c2261 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:21.626486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 51/97] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:41:21 2022] GET /identity/v3/role_assignments?user.id=067af36c9f9b401d9d35810237e9ba2e&scope.project.id=890c323b0dd14e24b001370bdb56e28d&role.id=824f5fe1f02b447382190a8c236c27db => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:22.931669 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6e98a-00c4-41f2-8094-985264c19325 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:22.932221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6e98a-00c4-41f2-8094-985264c19325 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:22.932221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6e98a-00c4-41f2-8094-985264c19325 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:22.933824 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 47/98] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:22.944235 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01302cbd-2b64-44c8-b7a5-bd81aa1ecbe0 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:22.944690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01302cbd-2b64-44c8-b7a5-bd81aa1ecbe0 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:22.944762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01302cbd-2b64-44c8-b7a5-bd81aa1ecbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:22.971638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-01302cbd-2b64-44c8-b7a5-bd81aa1ecbe0 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:22.972827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-01302cbd-2b64-44c8-b7a5-bd81aa1ecbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:23.071073 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 52/99] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:23.156954 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382828cf-8867-47be-9229-0104f39caaec None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:23.157289 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382828cf-8867-47be-9229-0104f39caaec None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:23.157911 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382828cf-8867-47be-9229-0104f39caaec None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:23.158987 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 48/100] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:23.165597 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d672de0-fe51-49c9-b62e-e887e903cd46 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:23.166130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d672de0-fe51-49c9-b62e-e887e903cd46 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:23.166593 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d672de0-fe51-49c9-b62e-e887e903cd46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:23.195694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-4d672de0-fe51-49c9-b62e-e887e903cd46 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:23.197156 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4d672de0-fe51-49c9-b62e-e887e903cd46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:23.278258 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 53/101] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:23.527613 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382828cf-8867-47be-9229-0104f39caaec None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:23.584476 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:23.653861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382828cf-8867-47be-9229-0104f39caaec None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:23.689422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dda1df-a963-48ce-ba09-010774dbbe98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:23.690370 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dda1df-a963-48ce-ba09-010774dbbe98 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:23.690950 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dda1df-a963-48ce-ba09-010774dbbe98 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:23.691433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dda1df-a963-48ce-ba09-010774dbbe98 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:23.703143 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dda1df-a963-48ce-ba09-010774dbbe98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:23.708872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-56dda1df-a963-48ce-ba09-010774dbbe98 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 23:41:23.708872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:23.708872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:23.708872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:23.708872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:23.708872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:23.708872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:23.708872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:23.708872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:23.708872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:23.708872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:23.708872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:23.708872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:23.708872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:23.708872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:23.708872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:23.709686 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:23.709686 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:23.709686 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:23.709686 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:23.709686 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:23.709686 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:23.709686 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:23.709686 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:23.709686 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:23.709686 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:23.709686 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:23.709686 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:23.709686 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:23.709686 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:23.709686 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:23.709686 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:23.710523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:23.710523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:23.710523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:23.710523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:23.710523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:23.710523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:23.710523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:23.710523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:23.710523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:23.710523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:23.710523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 23:41:23.710523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:41:23.712726 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 49/102] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:23.719089 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d672de0-fe51-49c9-b62e-e887e903cd46 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:23.732548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33393ac-b564-4575-b223-287743aca6b6 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:23.734862 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33393ac-b564-4575-b223-287743aca6b6 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:23.735339 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33393ac-b564-4575-b223-287743aca6b6 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:23.736924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 50/103] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:23.743793 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c321bc-4cb8-400b-b0d9-2fdbc61cc73d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:23.744363 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c321bc-4cb8-400b-b0d9-2fdbc61cc73d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:23.744848 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c321bc-4cb8-400b-b0d9-2fdbc61cc73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:23.776609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-70c321bc-4cb8-400b-b0d9-2fdbc61cc73d None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:23.780926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-70c321bc-4cb8-400b-b0d9-2fdbc61cc73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:23.814422 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:23.907704 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d672de0-fe51-49c9-b62e-e887e903cd46 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:23.911550 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 51/104] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:23.912473 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c321bc-4cb8-400b-b0d9-2fdbc61cc73d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:23.973745 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3bbdd-b83d-42e1-8b34-49469a6a7abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:23.974365 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3bbdd-b83d-42e1-8b34-49469a6a7abf None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:23.975920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3bbdd-b83d-42e1-8b34-49469a6a7abf None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:23.976413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:23.978785 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3bbdd-b83d-42e1-8b34-49469a6a7abf None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:23.979727 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3bbdd-b83d-42e1-8b34-49469a6a7abf None admin] RBAC: Adding query filter params (name=reader) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:23.981971 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3bbdd-b83d-42e1-8b34-49469a6a7abf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:23.985650 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3bbdd-b83d-42e1-8b34-49469a6a7abf None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:24.008133 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 54/105] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:41:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:24.014074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3bbdd-b83d-42e1-8b34-49469a6a7abf None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:24.127080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c321bc-4cb8-400b-b0d9-2fdbc61cc73d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:24.197448 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3bbdd-b83d-42e1-8b34-49469a6a7abf None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:24.201518 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd102e7a-4573-4853-a413-37cd569318b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:24.202405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd102e7a-4573-4853-a413-37cd569318b1 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:24.202937 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd102e7a-4573-4853-a413-37cd569318b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:24.204203 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd102e7a-4573-4853-a413-37cd569318b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:24.210560 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd102e7a-4573-4853-a413-37cd569318b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:24.212181 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd102e7a-4573-4853-a413-37cd569318b1 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:24.213895 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 52/106] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:41:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:24.218353 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43316c-e5fe-4a69-902e-6a2db2b2e486 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:24.219158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43316c-e5fe-4a69-902e-6a2db2b2e486 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:24.219569 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43316c-e5fe-4a69-902e-6a2db2b2e486 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:24.220963 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 53/107] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:24.224018 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43316c-e5fe-4a69-902e-6a2db2b2e486 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:24.224493 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:24.263470 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd754dc-74dc-45c9-9e0f-fac900867bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:24.269359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd754dc-74dc-45c9-9e0f-fac900867bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:24.269797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd754dc-74dc-45c9-9e0f-fac900867bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:24.270207 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd754dc-74dc-45c9-9e0f-fac900867bc2 None admin] PATH_INFO: `/v3/users/9c30a93a9eea4ac188b9d8e577b934f8` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:24.291674 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd754dc-74dc-45c9-9e0f-fac900867bc2 None admin] RBAC: Authorizing `identity:get_user(user_id=9c30a93a9eea4ac188b9d8e577b934f8)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:24.293548 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd754dc-74dc-45c9-9e0f-fac900867bc2 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:24.296049 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 55/108] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:24 2022] GET /identity/v3/users/9c30a93a9eea4ac188b9d8e577b934f8 => generated 289 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:24.297512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f89fe5-6488-4b7f-b557-35f778353f2c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:24.298018 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f89fe5-6488-4b7f-b557-35f778353f2c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:24.298482 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f89fe5-6488-4b7f-b557-35f778353f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:24.326807 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b3f89fe5-6488-4b7f-b557-35f778353f2c None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:24.328795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b3f89fe5-6488-4b7f-b557-35f778353f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:24.376500 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:24.380786 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43316c-e5fe-4a69-902e-6a2db2b2e486 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:24.414136 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 56/109] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:24.415686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f89fe5-6488-4b7f-b557-35f778353f2c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:24.422894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a77abf-ad39-4233-95b4-9f2a8f492de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:24.423882 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a77abf-ad39-4233-95b4-9f2a8f492de4 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:24.424497 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a77abf-ad39-4233-95b4-9f2a8f492de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:24.425521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a77abf-ad39-4233-95b4-9f2a8f492de4 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:24.426801 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a77abf-ad39-4233-95b4-9f2a8f492de4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:24.429061 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a77abf-ad39-4233-95b4-9f2a8f492de4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:24.525886 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:24.530515 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f89fe5-6488-4b7f-b557-35f778353f2c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:24.536384 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 54/110] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Sep 10 23:41:24 2022] POST /identity/v3/projects => generated 302 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:41:24.537804 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a77abf-ad39-4233-95b4-9f2a8f492de4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:24.581471 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757d5cd-a472-4c4f-b5d0-69e94d8c411c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:24.584138 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757d5cd-a472-4c4f-b5d0-69e94d8c411c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:24.584435 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757d5cd-a472-4c4f-b5d0-69e94d8c411c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:24.584711 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757d5cd-a472-4c4f-b5d0-69e94d8c411c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:24.611583 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0757d5cd-a472-4c4f-b5d0-69e94d8c411c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:24.628311 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-0757d5cd-a472-4c4f-b5d0-69e94d8c411c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:24.628311 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:24.628311 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:24.628311 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:24.628311 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:24.628311 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:24.628311 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:24.628311 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:24.628311 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:24.628311 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:24.628311 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:24.628311 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:24.628311 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:24.628311 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:24.628311 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:24.628311 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:24.629225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:24.629225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:24.629225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:24.629225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:24.629225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:24.629225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:24.629225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:24.629225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:24.629225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:24.629225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:24.629225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:24.629225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:24.629225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:24.629225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:24.629225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:24.629225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:24.629971 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:24.629971 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:24.629971 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:24.629971 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:24.629971 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:24.629971 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:24.629971 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:24.629971 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:24.629971 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:24.629971 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:24.629971 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:24.629971 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:41:24.629971 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 57/111] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:24.641729 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757d5cd-a472-4c4f-b5d0-69e94d8c411c None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:24.786324 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a77abf-ad39-4233-95b4-9f2a8f492de4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:24.822742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757d5cd-a472-4c4f-b5d0-69e94d8c411c None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:24.826778 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d375d6-c75f-460b-9346-904794a1290f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:24.827403 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d375d6-c75f-460b-9346-904794a1290f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:24.827685 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d375d6-c75f-460b-9346-904794a1290f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:24.827958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d375d6-c75f-460b-9346-904794a1290f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:24.856134 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:24.864190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd6696-ec3c-4fd7-9ef4-545fed7ac515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:24.864801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd6696-ec3c-4fd7-9ef4-545fed7ac515 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:24.865073 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd6696-ec3c-4fd7-9ef4-545fed7ac515 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:24.865339 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd6696-ec3c-4fd7-9ef4-545fed7ac515 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:24.866014 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd6696-ec3c-4fd7-9ef4-545fed7ac515 None admin] RBAC: Adding query filter params (name=member) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:24.866297 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd6696-ec3c-4fd7-9ef4-545fed7ac515 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:24.868648 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd6696-ec3c-4fd7-9ef4-545fed7ac515 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:24.878176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 58/112] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:41:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:24.880272 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929d895-3c3e-41b7-b5e6-ced95a679a73 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:24.880272 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929d895-3c3e-41b7-b5e6-ced95a679a73 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:24.880272 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929d895-3c3e-41b7-b5e6-ced95a679a73 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:24.881877 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 59/113] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:24.882313 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929d895-3c3e-41b7-b5e6-ced95a679a73 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:24.914518 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:24.914728 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d375d6-c75f-460b-9346-904794a1290f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:24.914728 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d375d6-c75f-460b-9346-904794a1290f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:24.991181 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929d895-3c3e-41b7-b5e6-ced95a679a73 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:25.043006 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 55/114] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:41:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:41:25.043511 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb3583-0d46-48f5-b3a0-bd4cc90ad9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:25.044099 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb3583-0d46-48f5-b3a0-bd4cc90ad9fa None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:25.044380 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb3583-0d46-48f5-b3a0-bd4cc90ad9fa None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:25.045289 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb3583-0d46-48f5-b3a0-bd4cc90ad9fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:25.045497 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d375d6-c75f-460b-9346-904794a1290f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:25.059171 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffb3583-0d46-48f5-b3a0-bd4cc90ad9fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:25.064611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-6ffb3583-0d46-48f5-b3a0-bd4cc90ad9fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:41:25.064611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:25.064611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:25.064611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:25.064611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:25.064611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:25.064611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:25.064611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:25.064611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:25.064611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:25.064611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:25.064611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:25.064611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:25.064611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:25.064611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:25.064611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:25.065449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:25.065449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:25.065449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:25.065449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:25.065449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:25.065449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:25.065449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:25.065449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:25.065449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:25.065449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:25.065449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:25.065449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:25.065449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:25.065449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:25.065449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:25.065449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:25.066286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:25.066286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:25.066286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:25.066286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:25.066286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:25.066286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:25.066286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:25.066286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:25.066286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:25.066286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:25.066286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:41:25.066286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:41:25.066286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 60/115] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:41:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:25.070462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe2bd5-7105-403a-8e23-60eb7820de6d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:25.070958 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe2bd5-7105-403a-8e23-60eb7820de6d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:25.071426 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe2bd5-7105-403a-8e23-60eb7820de6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:25.097032 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-29fe2bd5-7105-403a-8e23-60eb7820de6d None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:25.098611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-29fe2bd5-7105-403a-8e23-60eb7820de6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:25.110111 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:25.164923 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d375d6-c75f-460b-9346-904794a1290f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:25.174191 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 61/116] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:25.175633 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb8b50-2b48-4eef-a29c-9e4af082e1aa None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:25.176016 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb8b50-2b48-4eef-a29c-9e4af082e1aa None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:25.176555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb8b50-2b48-4eef-a29c-9e4af082e1aa None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:25.181917 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 62/117] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:25.181917 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb8b50-2b48-4eef-a29c-9e4af082e1aa None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:25.194000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:25.210271 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f31237-e177-468f-b963-e6d7aa6943f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:25.211157 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f31237-e177-468f-b963-e6d7aa6943f7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:25.211601 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f31237-e177-468f-b963-e6d7aa6943f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:25.212022 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f31237-e177-468f-b963-e6d7aa6943f7 None admin] PATH_INFO: `/v3/users/067af36c9f9b401d9d35810237e9ba2e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:25.213319 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f31237-e177-468f-b963-e6d7aa6943f7 None admin] RBAC: Authorizing `identity:get_user(user_id=067af36c9f9b401d9d35810237e9ba2e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:25.214963 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f31237-e177-468f-b963-e6d7aa6943f7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:25.218229 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 56/118] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:25 2022] GET /identity/v3/users/067af36c9f9b401d9d35810237e9ba2e => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:25.223833 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f31237-e177-468f-b963-e6d7aa6943f7 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:25.315561 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb8b50-2b48-4eef-a29c-9e4af082e1aa None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:25.343026 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f31237-e177-468f-b963-e6d7aa6943f7 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:25.362808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ba01c-c30c-427f-bed4-c6fd50cd4eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:25.363596 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ba01c-c30c-427f-bed4-c6fd50cd4eba None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:25.363973 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ba01c-c30c-427f-bed4-c6fd50cd4eba None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:25.364323 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ba01c-c30c-427f-bed4-c6fd50cd4eba None admin] PATH_INFO: `/v3/projects/59ae2524fa884ab7a9682a93d368b500` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:25.365881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:25.367117 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ba01c-c30c-427f-bed4-c6fd50cd4eba None admin] RBAC: Authorizing `identity:get_project(project_id=59ae2524fa884ab7a9682a93d368b500)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:25.369015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ba01c-c30c-427f-bed4-c6fd50cd4eba None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:25.370768 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 63/119] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:25 2022] GET /identity/v3/projects/59ae2524fa884ab7a9682a93d368b500 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:25.374905 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc4bc5-1d77-46e6-acae-14e465de71a9 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:25.374905 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc4bc5-1d77-46e6-acae-14e465de71a9 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:25.374905 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc4bc5-1d77-46e6-acae-14e465de71a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:25.392045 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db21eea-cfd8-42c3-9176-480757d5b95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:25.393066 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db21eea-cfd8-42c3-9176-480757d5b95e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:25.393621 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db21eea-cfd8-42c3-9176-480757d5b95e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:25.394129 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db21eea-cfd8-42c3-9176-480757d5b95e None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:25.395074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db21eea-cfd8-42c3-9176-480757d5b95e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:25.395683 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db21eea-cfd8-42c3-9176-480757d5b95e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:25.396804 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-e4fc4bc5-1d77-46e6-acae-14e465de71a9 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:25.397693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db21eea-cfd8-42c3-9176-480757d5b95e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:25.398397 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e4fc4bc5-1d77-46e6-acae-14e465de71a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:25.410422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 57/120] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:25.411718 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db21eea-cfd8-42c3-9176-480757d5b95e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:25.476392 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 64/121] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:25.477028 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc4bc5-1d77-46e6-acae-14e465de71a9 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:25.493796 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:25.526291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db21eea-cfd8-42c3-9176-480757d5b95e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:25.564062 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba91c5e-5abb-4950-8582-72357218ff16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:25.565150 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba91c5e-5abb-4950-8582-72357218ff16 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:25.565692 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba91c5e-5abb-4950-8582-72357218ff16 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:25.566194 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba91c5e-5abb-4950-8582-72357218ff16 None admin] PATH_INFO: `/v3/projects/59ae2524fa884ab7a9682a93d368b500` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:25.568176 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba91c5e-5abb-4950-8582-72357218ff16 None admin] RBAC: Authorizing `identity:get_project(project_id=59ae2524fa884ab7a9682a93d368b500)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:25.570749 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba91c5e-5abb-4950-8582-72357218ff16 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:25.573660 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 58/122] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:25 2022] GET /identity/v3/projects/59ae2524fa884ab7a9682a93d368b500 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:25.574796 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba91c5e-5abb-4950-8582-72357218ff16 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:25.597845 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc4bc5-1d77-46e6-acae-14e465de71a9 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:25.635433 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fee1c6-8ab9-4c56-80db-430526b4af0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:25.636068 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fee1c6-8ab9-4c56-80db-430526b4af0d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:25.636343 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fee1c6-8ab9-4c56-80db-430526b4af0d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:25.636610 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fee1c6-8ab9-4c56-80db-430526b4af0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:25.637331 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fee1c6-8ab9-4c56-80db-430526b4af0d None admin] RBAC: Adding query filter params (role.id=900ce80935da4fa59be9334c5c8e0083, scope.project.id=59ae2524fa884ab7a9682a93d368b500, user.id=9c30a93a9eea4ac188b9d8e577b934f8) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:25.637656 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fee1c6-8ab9-4c56-80db-430526b4af0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:25.639173 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fee1c6-8ab9-4c56-80db-430526b4af0d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:25.640751 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:25.646070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 65/123] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:41:25 2022] GET /identity/v3/role_assignments?user.id=9c30a93a9eea4ac188b9d8e577b934f8&scope.project.id=59ae2524fa884ab7a9682a93d368b500&role.id=900ce80935da4fa59be9334c5c8e0083 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:25.647241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fee1c6-8ab9-4c56-80db-430526b4af0d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:25.711946 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba91c5e-5abb-4950-8582-72357218ff16 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:25.733299 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:25.750780 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a523738-934c-4622-9f35-e6186199a7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:25.751868 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a523738-934c-4622-9f35-e6186199a7c9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:25.752341 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a523738-934c-4622-9f35-e6186199a7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:25.752829 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a523738-934c-4622-9f35-e6186199a7c9 None admin] PATH_INFO: `/v3/users/5243f89b8bf44f868ec5406d662b5cd9` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:25.754180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a523738-934c-4622-9f35-e6186199a7c9 None admin] RBAC: Authorizing `identity:get_user(user_id=5243f89b8bf44f868ec5406d662b5cd9)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:25.755912 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a523738-934c-4622-9f35-e6186199a7c9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:25.757855 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 59/124] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:25 2022] GET /identity/v3/users/5243f89b8bf44f868ec5406d662b5cd9 => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:25.758978 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a523738-934c-4622-9f35-e6186199a7c9 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:25.770510 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fee1c6-8ab9-4c56-80db-430526b4af0d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:25.819803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8e0da-4402-416b-a0c7-19428b41b165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:25.820610 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8e0da-4402-416b-a0c7-19428b41b165 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:25.821001 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8e0da-4402-416b-a0c7-19428b41b165 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:25.821891 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8e0da-4402-416b-a0c7-19428b41b165 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:25.837391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b8e0da-4402-416b-a0c7-19428b41b165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:25.845165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-07b8e0da-4402-416b-a0c7-19428b41b165 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 23:41:25.845165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:25.845165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:25.845165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:25.845165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:25.845165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:25.845165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:25.845165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:25.845165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:25.845165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:25.845165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:25.845165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:25.845165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:25.845165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:25.845165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:25.845165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:25.846109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:25.846109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:25.846109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:25.846109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:25.846109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:25.846109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:25.846109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:25.846109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:25.846109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:25.846109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:25.846109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:25.846109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:25.846109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:25.846109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:25.846109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:25.846109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:25.846902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:25.846902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:25.846902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:25.846902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:25.846902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:25.846902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:25.846902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:25.846902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:25.846902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:25.846902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:25.846902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 23:41:25.846902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:41:25.847489 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 66/125] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Sep 10 23:41:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:25.848314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8e0da-4402-416b-a0c7-19428b41b165 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:25.880571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a523738-934c-4622-9f35-e6186199a7c9 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:25.902077 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:25.922529 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480992c2-69ce-404b-a9f5-6f4f453869bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:25.923265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480992c2-69ce-404b-a9f5-6f4f453869bb None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:25.923541 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480992c2-69ce-404b-a9f5-6f4f453869bb None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:25.923829 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480992c2-69ce-404b-a9f5-6f4f453869bb None admin] PATH_INFO: `/v3/projects/59ae2524fa884ab7a9682a93d368b500/users/067af36c9f9b401d9d35810237e9ba2e/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:25.925724 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480992c2-69ce-404b-a9f5-6f4f453869bb None admin] RBAC: Authorizing `identity:create_grant(project_id=59ae2524fa884ab7a9682a93d368b500, user_id=067af36c9f9b401d9d35810237e9ba2e, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:25.927224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480992c2-69ce-404b-a9f5-6f4f453869bb None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:25.982147 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8e0da-4402-416b-a0c7-19428b41b165 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:26.011503 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 60/126] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Sep 10 23:41:25 2022] PUT /identity/v3/projects/59ae2524fa884ab7a9682a93d368b500/users/067af36c9f9b401d9d35810237e9ba2e/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:41:26.013922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480992c2-69ce-404b-a9f5-6f4f453869bb None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:26.016641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:26.023784 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d99c0b-dd05-48d1-9910-d58eec887362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:26.024595 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d99c0b-dd05-48d1-9910-d58eec887362 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:26.024917 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d99c0b-dd05-48d1-9910-d58eec887362 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:26.025379 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d99c0b-dd05-48d1-9910-d58eec887362 None admin] PATH_INFO: `/v3/projects/59ae2524fa884ab7a9682a93d368b500` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:26.026938 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d99c0b-dd05-48d1-9910-d58eec887362 None admin] RBAC: Authorizing `identity:get_project(project_id=59ae2524fa884ab7a9682a93d368b500)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:26.028658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d99c0b-dd05-48d1-9910-d58eec887362 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:26.035895 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 67/127] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:25 2022] GET /identity/v3/projects/59ae2524fa884ab7a9682a93d368b500 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:26.035895 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d99c0b-dd05-48d1-9910-d58eec887362 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:26.151489 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480992c2-69ce-404b-a9f5-6f4f453869bb None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:26.179498 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d99c0b-dd05-48d1-9910-d58eec887362 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:26.194435 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c111a1f-bf2d-4671-bd27-50b11ad26e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:26.195532 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c111a1f-bf2d-4671-bd27-50b11ad26e7e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:26.196217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c111a1f-bf2d-4671-bd27-50b11ad26e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:26.196682 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c111a1f-bf2d-4671-bd27-50b11ad26e7e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:26.214990 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c111a1f-bf2d-4671-bd27-50b11ad26e7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:26.227688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-1c111a1f-bf2d-4671-bd27-50b11ad26e7e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:26.227688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:26.227688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:26.227688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:26.227688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:26.227688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:26.227688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:26.227688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:26.227688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:26.227688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:26.227688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:26.227688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:26.227688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:26.227688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:26.227688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:26.227688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:26.228621 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:26.228621 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:26.228621 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:26.228621 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:26.228621 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:26.228621 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:26.228621 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:26.228621 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:26.228621 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:26.228621 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:26.228621 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:26.228621 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:26.228621 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:26.228621 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:26.228621 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:26.228621 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:26.229789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:26.229789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:26.229789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:26.229789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:26.229789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:26.229789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:26.229789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:26.229789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:26.229789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:26.229789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:26.229789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:26.229789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:41:26.229789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 61/128] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:26.230402 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c111a1f-bf2d-4671-bd27-50b11ad26e7e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:26.234942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacbfa59-3430-4267-aab7-7b92197e5350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:26.235605 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacbfa59-3430-4267-aab7-7b92197e5350 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:26.235887 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacbfa59-3430-4267-aab7-7b92197e5350 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:26.236160 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacbfa59-3430-4267-aab7-7b92197e5350 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:26.236833 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacbfa59-3430-4267-aab7-7b92197e5350 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:26.237134 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacbfa59-3430-4267-aab7-7b92197e5350 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:26.240101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacbfa59-3430-4267-aab7-7b92197e5350 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:26.257639 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 68/129] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:41:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:26.260031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacbfa59-3430-4267-aab7-7b92197e5350 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:26.314034 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:26.374995 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:26.404572 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c111a1f-bf2d-4671-bd27-50b11ad26e7e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:26.404765 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacbfa59-3430-4267-aab7-7b92197e5350 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:26.442858 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544f65f-80e0-447f-aded-6896587ce1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:26.443655 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544f65f-80e0-447f-aded-6896587ce1be None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:26.443947 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544f65f-80e0-447f-aded-6896587ce1be None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:26.444225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544f65f-80e0-447f-aded-6896587ce1be None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:26.445224 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d544f65f-80e0-447f-aded-6896587ce1be None admin] RBAC: Adding query filter params (name=member) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:26.445793 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d544f65f-80e0-447f-aded-6896587ce1be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:26.447228 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d544f65f-80e0-447f-aded-6896587ce1be None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:26.449919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3330fd56-e227-4393-af9b-186da3fbaa36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:26.451548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330fd56-e227-4393-af9b-186da3fbaa36 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:26.451950 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330fd56-e227-4393-af9b-186da3fbaa36 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:26.452317 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330fd56-e227-4393-af9b-186da3fbaa36 None admin] PATH_INFO: `/v3/projects/59ae2524fa884ab7a9682a93d368b500/users/5243f89b8bf44f868ec5406d662b5cd9/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:26.455147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3330fd56-e227-4393-af9b-186da3fbaa36 None admin] RBAC: Authorizing `identity:create_grant(project_id=59ae2524fa884ab7a9682a93d368b500, user_id=5243f89b8bf44f868ec5406d662b5cd9, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:26.456708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3330fd56-e227-4393-af9b-186da3fbaa36 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:26.456892 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 69/130] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:41:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:26.457731 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544f65f-80e0-447f-aded-6896587ce1be None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:26.567183 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544f65f-80e0-447f-aded-6896587ce1be None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:26.578795 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 62/131] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Sep 10 23:41:26 2022] PUT /identity/v3/projects/59ae2524fa884ab7a9682a93d368b500/users/5243f89b8bf44f868ec5406d662b5cd9/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:41:26.583676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3330fd56-e227-4393-af9b-186da3fbaa36 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:26.613734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:26.625278 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d9aef7-7eb5-484a-a868-3f24d67f2dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:26.625937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9aef7-7eb5-484a-a868-3f24d67f2dba None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:26.626214 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9aef7-7eb5-484a-a868-3f24d67f2dba None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:26.626485 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9aef7-7eb5-484a-a868-3f24d67f2dba None admin] PATH_INFO: `/v3/users/067af36c9f9b401d9d35810237e9ba2e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:26.629711 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d9aef7-7eb5-484a-a868-3f24d67f2dba None admin] RBAC: Authorizing `identity:get_user(user_id=067af36c9f9b401d9d35810237e9ba2e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:26.629711 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d9aef7-7eb5-484a-a868-3f24d67f2dba None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:26.631018 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 70/132] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:26 2022] GET /identity/v3/users/067af36c9f9b401d9d35810237e9ba2e => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:26.636971 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d9aef7-7eb5-484a-a868-3f24d67f2dba None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:26.749690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d9aef7-7eb5-484a-a868-3f24d67f2dba None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:26.786493 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e3786-4452-470a-9de1-312ff0e7cf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:26.787174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e3786-4452-470a-9de1-312ff0e7cf0e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:26.787458 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e3786-4452-470a-9de1-312ff0e7cf0e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:26.787736 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e3786-4452-470a-9de1-312ff0e7cf0e None admin] PATH_INFO: `/v3/projects/59ae2524fa884ab7a9682a93d368b500` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:26.788901 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e3786-4452-470a-9de1-312ff0e7cf0e None admin] RBAC: Authorizing `identity:get_project(project_id=59ae2524fa884ab7a9682a93d368b500)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:26.790529 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e3786-4452-470a-9de1-312ff0e7cf0e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:26.792398 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 71/133] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:26 2022] GET /identity/v3/projects/59ae2524fa884ab7a9682a93d368b500 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:41:26.796878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3330fd56-e227-4393-af9b-186da3fbaa36 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:26.798536 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:26.799510 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e3786-4452-470a-9de1-312ff0e7cf0e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:26.869086 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:26.875462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e1d11-d8ef-4486-a10b-f8620a4180d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:26.876118 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e1d11-d8ef-4486-a10b-f8620a4180d5 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:26.876400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e1d11-d8ef-4486-a10b-f8620a4180d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:26.876677 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e1d11-d8ef-4486-a10b-f8620a4180d5 None admin] PATH_INFO: `/v3/users/067af36c9f9b401d9d35810237e9ba2e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:26.881321 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e1d11-d8ef-4486-a10b-f8620a4180d5 None admin] RBAC: Authorizing `identity:get_user(user_id=067af36c9f9b401d9d35810237e9ba2e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:26.881321 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e1d11-d8ef-4486-a10b-f8620a4180d5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:26.881321 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 63/134] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:26 2022] GET /identity/v3/users/067af36c9f9b401d9d35810237e9ba2e => generated 275 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:26.900157 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e1d11-d8ef-4486-a10b-f8620a4180d5 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:26.916522 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e3786-4452-470a-9de1-312ff0e7cf0e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:26.954784 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be2296-9d05-45b6-b7d4-b4f0b07b4eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:26.955449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be2296-9d05-45b6-b7d4-b4f0b07b4eef None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:26.955725 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be2296-9d05-45b6-b7d4-b4f0b07b4eef None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:26.955993 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be2296-9d05-45b6-b7d4-b4f0b07b4eef None admin] PATH_INFO: `/v3/projects/59ae2524fa884ab7a9682a93d368b500/users/067af36c9f9b401d9d35810237e9ba2e/roles/ff2ed192587843e5b10b5a35949e4c5f` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:26.958298 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be2296-9d05-45b6-b7d4-b4f0b07b4eef None admin] RBAC: Authorizing `identity:create_grant(project_id=59ae2524fa884ab7a9682a93d368b500, user_id=067af36c9f9b401d9d35810237e9ba2e, role_id=ff2ed192587843e5b10b5a35949e4c5f)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:26.960051 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be2296-9d05-45b6-b7d4-b4f0b07b4eef None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:27.025786 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e1d11-d8ef-4486-a10b-f8620a4180d5 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:27.042648 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 72/135] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Sep 10 23:41:26 2022] PUT /identity/v3/projects/59ae2524fa884ab7a9682a93d368b500/users/067af36c9f9b401d9d35810237e9ba2e/roles/ff2ed192587843e5b10b5a35949e4c5f => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:41:27.081057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd8840b-4223-41fc-9e9c-cab8e819286e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:27.081757 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8840b-4223-41fc-9e9c-cab8e819286e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:27.082037 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8840b-4223-41fc-9e9c-cab8e819286e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:27.082309 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8840b-4223-41fc-9e9c-cab8e819286e None admin] PATH_INFO: `/v3/projects/890c323b0dd14e24b001370bdb56e28d` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:27.083499 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd8840b-4223-41fc-9e9c-cab8e819286e None admin] RBAC: Authorizing `identity:get_project(project_id=890c323b0dd14e24b001370bdb56e28d)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:27.085051 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd8840b-4223-41fc-9e9c-cab8e819286e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:27.087041 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 64/136] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:26 2022] GET /identity/v3/projects/890c323b0dd14e24b001370bdb56e28d => generated 312 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:27.093399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be2296-9d05-45b6-b7d4-b4f0b07b4eef None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:27.246462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be2296-9d05-45b6-b7d4-b4f0b07b4eef None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:27.274444 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:27.297473 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7726a57-ed9b-4512-963f-695ea59cb1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:27.297473 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7726a57-ed9b-4512-963f-695ea59cb1ab None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:27.297473 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7726a57-ed9b-4512-963f-695ea59cb1ab None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:27.297473 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7726a57-ed9b-4512-963f-695ea59cb1ab None admin] PATH_INFO: `/v3/projects/890c323b0dd14e24b001370bdb56e28d/users/067af36c9f9b401d9d35810237e9ba2e/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:27.298815 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7726a57-ed9b-4512-963f-695ea59cb1ab None admin] RBAC: Authorizing `identity:create_grant(project_id=890c323b0dd14e24b001370bdb56e28d, user_id=067af36c9f9b401d9d35810237e9ba2e, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:27.300281 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7726a57-ed9b-4512-963f-695ea59cb1ab None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:27.343209 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664d496-28a3-47e4-b0c0-c8c310b95e08 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:27.343726 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664d496-28a3-47e4-b0c0-c8c310b95e08 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:27.343726 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664d496-28a3-47e4-b0c0-c8c310b95e08 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:27.345289 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 65/137] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:41:27.351152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee082d-b350-4ff7-a637-7972cf848964 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:27.351435 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee082d-b350-4ff7-a637-7972cf848964 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:27.351711 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee082d-b350-4ff7-a637-7972cf848964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:27.378789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-3eee082d-b350-4ff7-a637-7972cf848964 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:27.380390 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3eee082d-b350-4ff7-a637-7972cf848964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:27.413364 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 73/138] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Sep 10 23:41:27 2022] PUT /identity/v3/projects/890c323b0dd14e24b001370bdb56e28d/users/067af36c9f9b401d9d35810237e9ba2e/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:41:27.423232 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:27.476216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 66/139] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:27.964798 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7726a57-ed9b-4512-963f-695ea59cb1ab None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:28.103507 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7726a57-ed9b-4512-963f-695ea59cb1ab None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:28.140692 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3094fba-cf94-483d-85dc-fa4ceaf16a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:28.141347 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3094fba-cf94-483d-85dc-fa4ceaf16a6d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:28.141628 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3094fba-cf94-483d-85dc-fa4ceaf16a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:28.141898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3094fba-cf94-483d-85dc-fa4ceaf16a6d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:28.143054 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3094fba-cf94-483d-85dc-fa4ceaf16a6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:28.144517 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3094fba-cf94-483d-85dc-fa4ceaf16a6d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:28.146110 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 74/140] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:41:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:28.152083 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee082d-b350-4ff7-a637-7972cf848964 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:28.292500 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee082d-b350-4ff7-a637-7972cf848964 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:28.352318 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e195470-06d9-42d0-abc8-2c1496e7c0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:28.352934 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e195470-06d9-42d0-abc8-2c1496e7c0c7 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:28.353205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e195470-06d9-42d0-abc8-2c1496e7c0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:28.353476 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e195470-06d9-42d0-abc8-2c1496e7c0c7 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:28.354134 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e195470-06d9-42d0-abc8-2c1496e7c0c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:28.355599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e195470-06d9-42d0-abc8-2c1496e7c0c7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:28.480423 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 67/141] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Sep 10 23:41:28 2022] POST /identity/v3/users => generated 283 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:41:28.712125 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea3559-3017-4137-89d7-a97bd72c905a None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:28.712494 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea3559-3017-4137-89d7-a97bd72c905a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:28.712687 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea3559-3017-4137-89d7-a97bd72c905a None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:28.714447 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 75/142] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:28.722896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1db12-8008-4ea3-8025-e365be21fd9e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:28.723210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1db12-8008-4ea3-8025-e365be21fd9e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:28.723462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1db12-8008-4ea3-8025-e365be21fd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:28.752552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-e6f1db12-8008-4ea3-8025-e365be21fd9e None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:28.753762 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e6f1db12-8008-4ea3-8025-e365be21fd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:28.772494 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:28.836810 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 68/143] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 23:41:29.309032 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d69e3-b4d3-4e8a-b821-8f25e4fba43d None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:29.309432 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d69e3-b4d3-4e8a-b821-8f25e4fba43d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:29.309633 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d69e3-b4d3-4e8a-b821-8f25e4fba43d None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:29.311579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 76/144] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:29.317390 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986ac92-4af6-4536-b143-1a3276a048ba None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:29.317688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986ac92-4af6-4536-b143-1a3276a048ba None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:29.317970 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986ac92-4af6-4536-b143-1a3276a048ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:29.341015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d69e3-b4d3-4e8a-b821-8f25e4fba43d None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:29.344394 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7986ac92-4af6-4536-b143-1a3276a048ba None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:29.345956 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7986ac92-4af6-4536-b143-1a3276a048ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:29.445147 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:29.520200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d69e3-b4d3-4e8a-b821-8f25e4fba43d None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:29.527000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 69/145] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:29.565196 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc259c-6e4f-4258-8ec8-605b77076f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:29.566061 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc259c-6e4f-4258-8ec8-605b77076f32 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:29.566477 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc259c-6e4f-4258-8ec8-605b77076f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:29.566896 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc259c-6e4f-4258-8ec8-605b77076f32 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:29.581552 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc259c-6e4f-4258-8ec8-605b77076f32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:29.587098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-90dc259c-6e4f-4258-8ec8-605b77076f32 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 23:41:29.587098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:29.587098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:29.587098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:29.587098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:29.587098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:29.587098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:29.587098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:29.587098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:29.587098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:29.587098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:29.587098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:29.587098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:29.587098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:29.587098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:29.587098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:29.588106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:29.588106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:29.588106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:29.588106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:29.588106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:29.588106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:29.588106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:29.588106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:29.588106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:29.588106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:29.588106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:29.588106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:29.588106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:29.588106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:29.588106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:29.588106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:29.589039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:29.589039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:29.589039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:29.589039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:29.589039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:29.589039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:29.589039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:29.589039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:29.589039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:29.589039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:29.589039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 23:41:29.589039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:41:29.590948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 77/146] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:29.596958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7986ac92-4af6-4536-b143-1a3276a048ba None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:29.716880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:29.741792 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7986ac92-4af6-4536-b143-1a3276a048ba None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:29.785606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5410f-39f8-4799-921c-765c0b18df56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:29.786220 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5410f-39f8-4799-921c-765c0b18df56 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:29.786503 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5410f-39f8-4799-921c-765c0b18df56 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:29.786814 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5410f-39f8-4799-921c-765c0b18df56 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:29.787509 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5410f-39f8-4799-921c-765c0b18df56 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:29.787813 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5410f-39f8-4799-921c-765c0b18df56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:29.789293 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5410f-39f8-4799-921c-765c0b18df56 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:29.800784 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 70/147] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:41:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:29.806022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc259c-6e4f-4258-8ec8-605b77076f32 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:29.946057 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc259c-6e4f-4258-8ec8-605b77076f32 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:29.959687 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:29.989063 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcba36d-4576-49b0-bc14-98722cd5311c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:29.990024 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcba36d-4576-49b0-bc14-98722cd5311c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:29.990489 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcba36d-4576-49b0-bc14-98722cd5311c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:29.990979 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcba36d-4576-49b0-bc14-98722cd5311c None admin] PATH_INFO: `/v3/users/9c30a93a9eea4ac188b9d8e577b934f8` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:29.995373 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcba36d-4576-49b0-bc14-98722cd5311c None admin] RBAC: Authorizing `identity:get_user(user_id=9c30a93a9eea4ac188b9d8e577b934f8)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:29.997091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcba36d-4576-49b0-bc14-98722cd5311c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:29.999406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 78/148] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:29 2022] GET /identity/v3/users/9c30a93a9eea4ac188b9d8e577b934f8 => generated 289 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:30.004734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5410f-39f8-4799-921c-765c0b18df56 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:30.060822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcba36d-4576-49b0-bc14-98722cd5311c None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:30.171196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5410f-39f8-4799-921c-765c0b18df56 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:30.223618 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcba36d-4576-49b0-bc14-98722cd5311c None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:30.261218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426d936-4147-462a-af0a-78f34c93bc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:30.261857 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426d936-4147-462a-af0a-78f34c93bc65 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:30.262134 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426d936-4147-462a-af0a-78f34c93bc65 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:30.262405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426d936-4147-462a-af0a-78f34c93bc65 None admin] PATH_INFO: `/v3/projects/59ae2524fa884ab7a9682a93d368b500` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:30.264030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3426d936-4147-462a-af0a-78f34c93bc65 None admin] RBAC: Authorizing `identity:get_project(project_id=59ae2524fa884ab7a9682a93d368b500)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:30.265748 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3426d936-4147-462a-af0a-78f34c93bc65 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:30.267247 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 71/149] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:30 2022] GET /identity/v3/projects/59ae2524fa884ab7a9682a93d368b500 => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:30.268487 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3bba0-c428-4146-aa3f-cd579ab0613c None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:30.268778 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3bba0-c428-4146-aa3f-cd579ab0613c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:30.269058 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3bba0-c428-4146-aa3f-cd579ab0613c None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:30.270278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 72/150] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:30.271332 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d3548-f2ca-4b83-bb2b-379cbed2851a None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:30.271611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d3548-f2ca-4b83-bb2b-379cbed2851a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:30.271885 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d3548-f2ca-4b83-bb2b-379cbed2851a None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:30.273208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 73/151] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:30.275844 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1fdeb6-91c0-45f4-9937-ff960df3ec14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:30.276764 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fdeb6-91c0-45f4-9937-ff960df3ec14 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:30.277255 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fdeb6-91c0-45f4-9937-ff960df3ec14 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:30.277710 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fdeb6-91c0-45f4-9937-ff960df3ec14 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:30.277890 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d3548-f2ca-4b83-bb2b-379cbed2851a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:30.291261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1fdeb6-91c0-45f4-9937-ff960df3ec14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:30.293291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:30.299949 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-cb1fdeb6-91c0-45f4-9937-ff960df3ec14 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:30.299949 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:30.299949 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:30.299949 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:30.299949 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:30.299949 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:30.299949 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:30.299949 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:30.299949 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:30.299949 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:30.299949 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:30.299949 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:30.299949 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:30.299949 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:30.299949 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:30.299949 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:30.300818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:30.300818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:30.300818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:30.300818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:30.300818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:30.300818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:30.300818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:30.300818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:30.300818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:30.300818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:30.300818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:30.300818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:30.300818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:30.300818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:30.300818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:30.300818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:30.301608 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:30.301608 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:30.301608 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:30.301608 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:30.301608 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:30.301608 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:30.301608 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:30.301608 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:30.301608 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:30.301608 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:30.301608 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:30.301608 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:41:30.303903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 79/152] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:30.305308 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78b433-0e15-4a05-99c7-50367959d1a2 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:30.314013 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78b433-0e15-4a05-99c7-50367959d1a2 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:30.314013 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78b433-0e15-4a05-99c7-50367959d1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:30.330054 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-ce78b433-0e15-4a05-99c7-50367959d1a2 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:30.331318 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ce78b433-0e15-4a05-99c7-50367959d1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:30.372884 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:30.409191 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d3548-f2ca-4b83-bb2b-379cbed2851a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:30.423655 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 80/153] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:30.425942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966aa8f9-9140-4ea1-9bd7-0cd3c844b3b6 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:30.426371 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966aa8f9-9140-4ea1-9bd7-0cd3c844b3b6 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:30.426803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966aa8f9-9140-4ea1-9bd7-0cd3c844b3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:30.451950 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa0b8d-f467-4a86-ae34-18bebc57d9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:30.452157 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-966aa8f9-9140-4ea1-9bd7-0cd3c844b3b6 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:30.452942 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa0b8d-f467-4a86-ae34-18bebc57d9e7 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:30.453236 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-966aa8f9-9140-4ea1-9bd7-0cd3c844b3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:30.453538 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa0b8d-f467-4a86-ae34-18bebc57d9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:30.453944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa0b8d-f467-4a86-ae34-18bebc57d9e7 None admin] PATH_INFO: `/v3/projects/59ae2524fa884ab7a9682a93d368b500/users/9c30a93a9eea4ac188b9d8e577b934f8/roles/900ce80935da4fa59be9334c5c8e0083` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:30.456824 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa0b8d-f467-4a86-ae34-18bebc57d9e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=59ae2524fa884ab7a9682a93d368b500, user_id=9c30a93a9eea4ac188b9d8e577b934f8, role_id=900ce80935da4fa59be9334c5c8e0083)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:30.458427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa0b8d-f467-4a86-ae34-18bebc57d9e7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:30.538748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 81/154] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:30.539566 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966aa8f9-9140-4ea1-9bd7-0cd3c844b3b6 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:30.547878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 74/155] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Sep 10 23:41:30 2022] PUT /identity/v3/projects/59ae2524fa884ab7a9682a93d368b500/users/9c30a93a9eea4ac188b9d8e577b934f8/roles/900ce80935da4fa59be9334c5c8e0083 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:41:30.672779 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966aa8f9-9140-4ea1-9bd7-0cd3c844b3b6 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:30.680870 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:30.714120 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b828de-6f04-449e-b40b-bd2ffab50f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:30.714768 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b828de-6f04-449e-b40b-bd2ffab50f65 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:30.715046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b828de-6f04-449e-b40b-bd2ffab50f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:30.715318 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b828de-6f04-449e-b40b-bd2ffab50f65 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:30.716000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b828de-6f04-449e-b40b-bd2ffab50f65 None admin] RBAC: Adding query filter params (name=member) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:30.716412 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b828de-6f04-449e-b40b-bd2ffab50f65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:30.718379 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b828de-6f04-449e-b40b-bd2ffab50f65 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:30.728839 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 82/156] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:41:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:30.736033 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa0b8d-f467-4a86-ae34-18bebc57d9e7 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:30.829090 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:30.874223 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa0b8d-f467-4a86-ae34-18bebc57d9e7 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:30.917176 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647fe90-6878-4404-baee-3ee2f9fa4755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:30.917823 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647fe90-6878-4404-baee-3ee2f9fa4755 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:30.918178 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647fe90-6878-4404-baee-3ee2f9fa4755 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:30.918455 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647fe90-6878-4404-baee-3ee2f9fa4755 None admin] PATH_INFO: `/v3/users/067af36c9f9b401d9d35810237e9ba2e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:30.923545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647fe90-6878-4404-baee-3ee2f9fa4755 None admin] RBAC: Authorizing `identity:get_user(user_id=067af36c9f9b401d9d35810237e9ba2e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:30.925344 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647fe90-6878-4404-baee-3ee2f9fa4755 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:30.928005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 75/157] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:30 2022] GET /identity/v3/users/067af36c9f9b401d9d35810237e9ba2e => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:30.934420 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b828de-6f04-449e-b40b-bd2ffab50f65 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:31.052070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b828de-6f04-449e-b40b-bd2ffab50f65 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:31.071170 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2b122-b66f-44d2-8237-8ddd7fb4c4ba None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:31.071471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2b122-b66f-44d2-8237-8ddd7fb4c4ba None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:31.071785 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2b122-b66f-44d2-8237-8ddd7fb4c4ba None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:31.073306 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 76/158] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:41:31.094549 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1174f-e38c-4e13-90a0-c2ae21e9913c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:31.094893 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1174f-e38c-4e13-90a0-c2ae21e9913c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:31.095091 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1174f-e38c-4e13-90a0-c2ae21e9913c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:31.120860 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da391e-c75c-42f7-800e-154d0861039b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:31.121482 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da391e-c75c-42f7-800e-154d0861039b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:31.121757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da391e-c75c-42f7-800e-154d0861039b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:31.122029 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da391e-c75c-42f7-800e-154d0861039b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:31.125997 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-3bb1174f-e38c-4e13-90a0-c2ae21e9913c None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:31.127419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3bb1174f-e38c-4e13-90a0-c2ae21e9913c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:31.166059 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:31.255177 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 77/159] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 23:41:31.256247 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb1174f-e38c-4e13-90a0-c2ae21e9913c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:31.272849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31da391e-c75c-42f7-800e-154d0861039b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:31.274642 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31da391e-c75c-42f7-800e-154d0861039b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:31.367132 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:31.417749 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb1174f-e38c-4e13-90a0-c2ae21e9913c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:31.454190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 83/160] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:41:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 10 23:41:31.455679 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da391e-c75c-42f7-800e-154d0861039b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:31.499319 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6345bf-defc-44ec-8c57-3ed5b31c8d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:31.499948 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6345bf-defc-44ec-8c57-3ed5b31c8d31 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:31.500222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6345bf-defc-44ec-8c57-3ed5b31c8d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:31.500489 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6345bf-defc-44ec-8c57-3ed5b31c8d31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:31.526951 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6345bf-defc-44ec-8c57-3ed5b31c8d31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:31.533179 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-9a6345bf-defc-44ec-8c57-3ed5b31c8d31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:41:31.533179 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:31.533179 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:31.533179 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:31.533179 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:31.533179 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:31.533179 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:31.533179 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:31.533179 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:31.533179 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:31.533179 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:31.533179 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:31.533179 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:31.533179 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:31.533179 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:31.533179 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:31.534036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:31.534036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:31.534036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:31.534036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:31.534036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:31.534036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:31.534036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:31.534036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:31.534036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:31.534036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:31.534036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:31.534036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:31.534036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:31.534036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:31.534036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:31.534036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:31.534903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:31.534903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:31.534903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:31.534903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:31.534903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:31.534903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:31.534903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:31.534903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:31.534903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:31.534903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:31.534903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:41:31.534903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:41:31.551842 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 78/161] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:41:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:31.552782 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6345bf-defc-44ec-8c57-3ed5b31c8d31 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:31.604643 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:31.702406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da391e-c75c-42f7-800e-154d0861039b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:31.773217 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b5df3-c935-443b-a53b-fb5371f3480a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:31.773859 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b5df3-c935-443b-a53b-fb5371f3480a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:31.774137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b5df3-c935-443b-a53b-fb5371f3480a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:31.774409 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b5df3-c935-443b-a53b-fb5371f3480a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:31.783952 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6345bf-defc-44ec-8c57-3ed5b31c8d31 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:31.797141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b5df3-c935-443b-a53b-fb5371f3480a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:31.802288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-3a6b5df3-c935-443b-a53b-fb5371f3480a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:31.802288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:31.802288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:31.802288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:31.802288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:31.802288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:31.802288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:31.802288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:31.802288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:31.802288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:31.802288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:31.802288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:31.802288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:31.802288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:31.802288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:31.802288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:31.803368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:31.803368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:31.803368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:31.803368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:31.803368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:31.803368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:31.803368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:31.803368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:31.803368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:31.803368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:31.803368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:31.803368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:31.803368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:31.803368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:31.803368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:31.803368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:31.804327 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:31.804327 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:31.804327 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:31.804327 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:31.804327 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:31.804327 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:31.804327 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:31.804327 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:31.804327 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:31.804327 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:31.804327 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:31.804327 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:41:31.806280 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 84/162] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:31.807519 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b5df3-c935-443b-a53b-fb5371f3480a None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:31.843581 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:31.864337 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade3901-17ea-4d50-991b-0e1cc5afad59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:31.864970 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade3901-17ea-4d50-991b-0e1cc5afad59 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:31.873314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade3901-17ea-4d50-991b-0e1cc5afad59 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:31.873314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade3901-17ea-4d50-991b-0e1cc5afad59 None admin] PATH_INFO: `/v3/projects/59ae2524fa884ab7a9682a93d368b500` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:31.873314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade3901-17ea-4d50-991b-0e1cc5afad59 None admin] RBAC: Authorizing `identity:get_project(project_id=59ae2524fa884ab7a9682a93d368b500)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:31.873314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade3901-17ea-4d50-991b-0e1cc5afad59 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:31.874577 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 79/163] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:31 2022] GET /identity/v3/projects/59ae2524fa884ab7a9682a93d368b500 => generated 298 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:31.875488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade3901-17ea-4d50-991b-0e1cc5afad59 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:31.924330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b5df3-c935-443b-a53b-fb5371f3480a None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:31.960808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb235036-9a91-406f-989c-2258e82aa382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:31.961696 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb235036-9a91-406f-989c-2258e82aa382 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:31.962143 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb235036-9a91-406f-989c-2258e82aa382 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:31.962578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb235036-9a91-406f-989c-2258e82aa382 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:31.963740 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb235036-9a91-406f-989c-2258e82aa382 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:31.965679 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:31.966151 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb235036-9a91-406f-989c-2258e82aa382 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:31.968367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb235036-9a91-406f-989c-2258e82aa382 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:31.977352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 85/164] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:31.978468 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb235036-9a91-406f-989c-2258e82aa382 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:32.065767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade3901-17ea-4d50-991b-0e1cc5afad59 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:32.150570 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb235036-9a91-406f-989c-2258e82aa382 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:32.161400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d508d-ccf8-446f-b7b4-abcca3bc2403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:32.162266 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d508d-ccf8-446f-b7b4-abcca3bc2403 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:32.162725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d508d-ccf8-446f-b7b4-abcca3bc2403 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:32.163153 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d508d-ccf8-446f-b7b4-abcca3bc2403 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:32.163976 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d508d-ccf8-446f-b7b4-abcca3bc2403 None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:32.164438 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d508d-ccf8-446f-b7b4-abcca3bc2403 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:32.165983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d508d-ccf8-446f-b7b4-abcca3bc2403 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:32.182526 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 80/165] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:41:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:32.183622 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d508d-ccf8-446f-b7b4-abcca3bc2403 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:32.225797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c42b2d-34db-4a5b-b2b8-1d6bbd3a46f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:32.226355 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c42b2d-34db-4a5b-b2b8-1d6bbd3a46f8 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:32.226639 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c42b2d-34db-4a5b-b2b8-1d6bbd3a46f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:32.226939 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c42b2d-34db-4a5b-b2b8-1d6bbd3a46f8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:32.251336 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c42b2d-34db-4a5b-b2b8-1d6bbd3a46f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:32.257015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-d6c42b2d-34db-4a5b-b2b8-1d6bbd3a46f8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 23:41:32.257015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:32.257015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:32.257015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:32.257015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:32.257015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:32.257015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:32.257015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:32.257015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:32.257015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:32.257015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:32.257015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:32.257015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:32.257015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:32.257015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:32.257015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:32.258141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:32.258141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:32.258141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:32.258141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:32.258141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:32.258141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:32.258141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:32.258141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:32.258141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:32.258141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:32.258141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:32.258141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:32.258141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:32.258141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:32.258141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:32.258141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:32.259166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:32.259166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:32.259166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:32.259166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:32.259166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:32.259166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:32.259166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:32.259166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:32.259166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:32.259166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:32.259166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 23:41:32.259166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:41:32.259166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 86/166] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Sep 10 23:41:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:32.259166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c42b2d-34db-4a5b-b2b8-1d6bbd3a46f8 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:32.332216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:32.362285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:32.393778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c42b2d-34db-4a5b-b2b8-1d6bbd3a46f8 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:32.434836 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d508d-ccf8-446f-b7b4-abcca3bc2403 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:32.440566 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef9006-b130-4f35-9b56-cb49416ac87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:32.441224 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef9006-b130-4f35-9b56-cb49416ac87e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:32.441507 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef9006-b130-4f35-9b56-cb49416ac87e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:32.441831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef9006-b130-4f35-9b56-cb49416ac87e None admin] PATH_INFO: `/v3/users/5243f89b8bf44f868ec5406d662b5cd9` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:32.451633 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef9006-b130-4f35-9b56-cb49416ac87e None admin] RBAC: Authorizing `identity:get_user(user_id=5243f89b8bf44f868ec5406d662b5cd9)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:32.453189 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef9006-b130-4f35-9b56-cb49416ac87e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:32.455304 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 87/167] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:32 2022] GET /identity/v3/users/5243f89b8bf44f868ec5406d662b5cd9 => generated 247 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:32.456156 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef9006-b130-4f35-9b56-cb49416ac87e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:32.480094 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849b296-9521-4b2b-b5a0-785d224919ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:32.481032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849b296-9521-4b2b-b5a0-785d224919ff None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:32.481522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849b296-9521-4b2b-b5a0-785d224919ff None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:32.482168 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849b296-9521-4b2b-b5a0-785d224919ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:32.483517 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849b296-9521-4b2b-b5a0-785d224919ff None admin] RBAC: Adding query filter params (role.id=824f5fe1f02b447382190a8c236c27db, scope.project.id=59ae2524fa884ab7a9682a93d368b500, user.id=067af36c9f9b401d9d35810237e9ba2e) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:32.484138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849b296-9521-4b2b-b5a0-785d224919ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:32.485882 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849b296-9521-4b2b-b5a0-785d224919ff None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:32.492706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 81/168] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:41:32 2022] GET /identity/v3/role_assignments?user.id=067af36c9f9b401d9d35810237e9ba2e&scope.project.id=59ae2524fa884ab7a9682a93d368b500&role.id=824f5fe1f02b447382190a8c236c27db => generated 608 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:32.494005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849b296-9521-4b2b-b5a0-785d224919ff None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:32.649248 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef9006-b130-4f35-9b56-cb49416ac87e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:32.671043 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:32.720640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849b296-9521-4b2b-b5a0-785d224919ff None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:32.726814 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b171444-6a9c-436a-afd4-09a2d2869552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:32.727754 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b171444-6a9c-436a-afd4-09a2d2869552 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:32.728212 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b171444-6a9c-436a-afd4-09a2d2869552 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:32.728671 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b171444-6a9c-436a-afd4-09a2d2869552 None admin] PATH_INFO: `/v3/users/067af36c9f9b401d9d35810237e9ba2e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:32.730018 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b171444-6a9c-436a-afd4-09a2d2869552 None admin] RBAC: Authorizing `identity:get_user(user_id=067af36c9f9b401d9d35810237e9ba2e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:32.732853 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b171444-6a9c-436a-afd4-09a2d2869552 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:32.736753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 88/169] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:32 2022] GET /identity/v3/users/067af36c9f9b401d9d35810237e9ba2e => generated 275 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:32.738096 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b171444-6a9c-436a-afd4-09a2d2869552 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:32.761277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:32.775356 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff045b33-f600-47ac-983d-16a911c6f0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:32.776227 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff045b33-f600-47ac-983d-16a911c6f0de None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:32.776644 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff045b33-f600-47ac-983d-16a911c6f0de None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:32.777048 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff045b33-f600-47ac-983d-16a911c6f0de None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:32.778113 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff045b33-f600-47ac-983d-16a911c6f0de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:32.778564 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff045b33-f600-47ac-983d-16a911c6f0de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:32.780775 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff045b33-f600-47ac-983d-16a911c6f0de None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:32.792674 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 82/170] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:41:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:32.793924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff045b33-f600-47ac-983d-16a911c6f0de None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:32.939728 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff045b33-f600-47ac-983d-16a911c6f0de None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:32.946640 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b171444-6a9c-436a-afd4-09a2d2869552 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:33.029783 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48771e04-5520-419c-a78d-7d5d1eeb6b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:33.030300 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48771e04-5520-419c-a78d-7d5d1eeb6b3e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:33.030576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48771e04-5520-419c-a78d-7d5d1eeb6b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:33.030883 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48771e04-5520-419c-a78d-7d5d1eeb6b3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:33.031374 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf1a4d-16d7-430e-bf16-1e8761ce965e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:33.032000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf1a4d-16d7-430e-bf16-1e8761ce965e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:33.032287 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf1a4d-16d7-430e-bf16-1e8761ce965e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:33.032556 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf1a4d-16d7-430e-bf16-1e8761ce965e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:33.129654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:33.168093 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:33.193069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf1a4d-16d7-430e-bf16-1e8761ce965e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:33.194568 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf1a4d-16d7-430e-bf16-1e8761ce965e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:33.219671 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48771e04-5520-419c-a78d-7d5d1eeb6b3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:33.221151 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48771e04-5520-419c-a78d-7d5d1eeb6b3e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:33.368344 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 83/171] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:41:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:41:33.369196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48771e04-5520-419c-a78d-7d5d1eeb6b3e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:33.434486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 89/172] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:41:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 697 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:41:33.435959 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691d200-46cb-4238-a3bc-e77548b59853 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:33.436197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691d200-46cb-4238-a3bc-e77548b59853 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:33.436465 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691d200-46cb-4238-a3bc-e77548b59853 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:33.437877 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 90/173] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:33.438396 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b691d200-46cb-4238-a3bc-e77548b59853 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:33.541339 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:33.560555 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48771e04-5520-419c-a78d-7d5d1eeb6b3e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:33.574910 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:33.600392 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b691d200-46cb-4238-a3bc-e77548b59853 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:33.604448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390809f2-95a6-412a-998c-e8e5363ef719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:33.605271 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390809f2-95a6-412a-998c-e8e5363ef719 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:33.605705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390809f2-95a6-412a-998c-e8e5363ef719 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:33.606127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390809f2-95a6-412a-998c-e8e5363ef719 None admin] PATH_INFO: `/v3/users/067af36c9f9b401d9d35810237e9ba2e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:33.608029 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390809f2-95a6-412a-998c-e8e5363ef719 None admin] RBAC: Authorizing `identity:get_user(user_id=067af36c9f9b401d9d35810237e9ba2e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:33.610100 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390809f2-95a6-412a-998c-e8e5363ef719 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:33.612742 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 84/174] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:33 2022] GET /identity/v3/users/067af36c9f9b401d9d35810237e9ba2e => generated 275 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:33.613762 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390809f2-95a6-412a-998c-e8e5363ef719 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:33.645287 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ec6d7-3619-4953-9ca8-04736ad7dcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:33.646462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ec6d7-3619-4953-9ca8-04736ad7dcfe None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:33.646955 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ec6d7-3619-4953-9ca8-04736ad7dcfe None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:33.647412 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ec6d7-3619-4953-9ca8-04736ad7dcfe None admin] PATH_INFO: `/v3/projects/890c323b0dd14e24b001370bdb56e28d` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:33.648994 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ec6d7-3619-4953-9ca8-04736ad7dcfe None admin] RBAC: Authorizing `identity:get_project(project_id=890c323b0dd14e24b001370bdb56e28d)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:33.650728 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ec6d7-3619-4953-9ca8-04736ad7dcfe None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:33.652816 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 91/175] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:33 2022] GET /identity/v3/projects/890c323b0dd14e24b001370bdb56e28d => generated 312 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:33.654503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf029d0-9ac5-4362-bc2a-3caa8ee3262a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:33.655058 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf029d0-9ac5-4362-bc2a-3caa8ee3262a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:33.655576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf029d0-9ac5-4362-bc2a-3caa8ee3262a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:33.689457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-fbf029d0-9ac5-4362-bc2a-3caa8ee3262a None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:33.691037 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-fbf029d0-9ac5-4362-bc2a-3caa8ee3262a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:33.745821 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390809f2-95a6-412a-998c-e8e5363ef719 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:33.782994 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 92/176] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:33.784967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591b0d5-88d2-4d7b-b0e2-23db08e10747 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:33.785369 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591b0d5-88d2-4d7b-b0e2-23db08e10747 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:33.786190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591b0d5-88d2-4d7b-b0e2-23db08e10747 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:33.787721 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 93/177] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:33.788757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591b0d5-88d2-4d7b-b0e2-23db08e10747 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:33.794431 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99405c-b582-4ce3-899c-74f9cd33f59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:33.795109 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99405c-b582-4ce3-899c-74f9cd33f59b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:33.795384 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99405c-b582-4ce3-899c-74f9cd33f59b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:33.795665 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99405c-b582-4ce3-899c-74f9cd33f59b None admin] PATH_INFO: `/v3/projects/59ae2524fa884ab7a9682a93d368b500` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:33.797024 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99405c-b582-4ce3-899c-74f9cd33f59b None admin] RBAC: Authorizing `identity:get_project(project_id=59ae2524fa884ab7a9682a93d368b500)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:33.798483 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99405c-b582-4ce3-899c-74f9cd33f59b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:33.800281 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 85/178] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:33 2022] GET /identity/v3/projects/59ae2524fa884ab7a9682a93d368b500 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:33.801428 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99405c-b582-4ce3-899c-74f9cd33f59b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:33.828163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:33.899184 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:33.921725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99405c-b582-4ce3-899c-74f9cd33f59b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:33.930864 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591b0d5-88d2-4d7b-b0e2-23db08e10747 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:33.962281 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b96e9-8820-46ea-a81a-a9ad8b4c221d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:33.962938 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b96e9-8820-46ea-a81a-a9ad8b4c221d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:33.963258 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b96e9-8820-46ea-a81a-a9ad8b4c221d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:33.963616 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b96e9-8820-46ea-a81a-a9ad8b4c221d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:33.964298 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b96e9-8820-46ea-a81a-a9ad8b4c221d None admin] RBAC: Adding query filter params (role.id=824f5fe1f02b447382190a8c236c27db, scope.project.id=890c323b0dd14e24b001370bdb56e28d, user.id=067af36c9f9b401d9d35810237e9ba2e) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:33.964604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b96e9-8820-46ea-a81a-a9ad8b4c221d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:33.966746 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b96e9-8820-46ea-a81a-a9ad8b4c221d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:33.979466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 86/179] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:41:33 2022] GET /identity/v3/role_assignments?user.id=067af36c9f9b401d9d35810237e9ba2e&scope.project.id=890c323b0dd14e24b001370bdb56e28d&role.id=824f5fe1f02b447382190a8c236c27db => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:33.979466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f869bd-575b-4986-858e-45099f838399 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:33.979466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f869bd-575b-4986-858e-45099f838399 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:33.979466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f869bd-575b-4986-858e-45099f838399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:33.984716 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c46c9c0-26a6-476a-b87d-197bf1fa0709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:33.985873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46c9c0-26a6-476a-b87d-197bf1fa0709 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:33.986286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46c9c0-26a6-476a-b87d-197bf1fa0709 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:33.986724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46c9c0-26a6-476a-b87d-197bf1fa0709 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:33.999884 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b9f869bd-575b-4986-858e-45099f838399 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:34.006691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b9f869bd-575b-4986-858e-45099f838399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:34.115213 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c46c9c0-26a6-476a-b87d-197bf1fa0709 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:34.116639 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c46c9c0-26a6-476a-b87d-197bf1fa0709 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:34.118652 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 87/180] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:34.120250 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37076b21-278e-4578-831c-cabb7a7e55bc None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:34.121471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37076b21-278e-4578-831c-cabb7a7e55bc None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:34.121749 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37076b21-278e-4578-831c-cabb7a7e55bc None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:34.123255 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 88/181] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:34.128228 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37076b21-278e-4578-831c-cabb7a7e55bc None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:34.151622 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:34.218122 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:34.243051 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 94/182] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:41:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:41:34.243908 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c46c9c0-26a6-476a-b87d-197bf1fa0709 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:34.271358 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37076b21-278e-4578-831c-cabb7a7e55bc None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:34.314727 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c4f0a-c125-4ed6-b974-3da3a8082d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:34.315546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c4f0a-c125-4ed6-b974-3da3a8082d89 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:34.315990 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c4f0a-c125-4ed6-b974-3da3a8082d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:34.316527 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c4f0a-c125-4ed6-b974-3da3a8082d89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:34.317309 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c4f0a-c125-4ed6-b974-3da3a8082d89 None admin] RBAC: Adding query filter params (role.id=9dce48a2b53c482d931dcd064b07744e, scope.project.id=59ae2524fa884ab7a9682a93d368b500, user.id=5243f89b8bf44f868ec5406d662b5cd9) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:34.317618 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c4f0a-c125-4ed6-b974-3da3a8082d89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:34.319229 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c4f0a-c125-4ed6-b974-3da3a8082d89 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:34.327660 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 89/183] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:41:34 2022] GET /identity/v3/role_assignments?user.id=5243f89b8bf44f868ec5406d662b5cd9&scope.project.id=59ae2524fa884ab7a9682a93d368b500&role.id=9dce48a2b53c482d931dcd064b07744e => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:34.328875 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a6e51-1cfc-43f7-af93-ecbb91ec1e4f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:34.329165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a6e51-1cfc-43f7-af93-ecbb91ec1e4f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:34.329418 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a6e51-1cfc-43f7-af93-ecbb91ec1e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:34.348994 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-cb0a6e51-1cfc-43f7-af93-ecbb91ec1e4f None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:34.350216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-cb0a6e51-1cfc-43f7-af93-ecbb91ec1e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:34.378425 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c46c9c0-26a6-476a-b87d-197bf1fa0709 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:34.427836 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f998c0-65c1-414f-9a17-ccbfa54f6ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:34.428816 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f998c0-65c1-414f-9a17-ccbfa54f6ffe None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:34.429315 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f998c0-65c1-414f-9a17-ccbfa54f6ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:34.429754 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f998c0-65c1-414f-9a17-ccbfa54f6ffe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:34.446725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 90/184] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:34.446725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a6e51-1cfc-43f7-af93-ecbb91ec1e4f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:34.451340 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f998c0-65c1-414f-9a17-ccbfa54f6ffe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:34.456943 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-40f998c0-65c1-414f-9a17-ccbfa54f6ffe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:41:34.456943 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:34.456943 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:34.456943 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:34.456943 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:34.456943 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:34.456943 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:34.456943 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:34.456943 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:34.456943 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:34.456943 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:34.456943 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:34.456943 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:34.456943 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:34.456943 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:34.456943 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:34.457874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:34.457874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:34.457874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:34.457874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:34.457874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:34.457874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:34.457874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:34.457874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:34.457874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:34.457874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:34.457874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:34.457874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:34.457874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:34.457874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:34.457874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:34.457874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:34.458638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:34.458638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:34.458638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:34.458638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:34.458638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:34.458638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:34.458638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:34.458638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:34.458638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:34.458638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:34.458638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:41:34.458638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:41:34.465920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 95/185] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:41:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:34.465920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d09e9-aac0-48a1-bd5d-45370e584eb2 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:34.465920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d09e9-aac0-48a1-bd5d-45370e584eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:34.465920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d09e9-aac0-48a1-bd5d-45370e584eb2 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:34.468396 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 96/186] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:34.471501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d09e9-aac0-48a1-bd5d-45370e584eb2 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:34.475036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:34.514430 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:34.586506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a6e51-1cfc-43f7-af93-ecbb91ec1e4f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:34.599472 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d09e9-aac0-48a1-bd5d-45370e584eb2 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:34.625283 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f89ba-a94a-4b1c-a138-94080d9c582b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:34.625917 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f89ba-a94a-4b1c-a138-94080d9c582b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:34.626189 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f89ba-a94a-4b1c-a138-94080d9c582b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:34.626455 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f89ba-a94a-4b1c-a138-94080d9c582b None admin] PATH_INFO: `/v3/projects/59ae2524fa884ab7a9682a93d368b500` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:34.627589 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f89ba-a94a-4b1c-a138-94080d9c582b None admin] RBAC: Authorizing `identity:get_project(project_id=59ae2524fa884ab7a9682a93d368b500)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:34.629347 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f89ba-a94a-4b1c-a138-94080d9c582b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:34.631248 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 91/187] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:34 2022] GET /identity/v3/projects/59ae2524fa884ab7a9682a93d368b500 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:34.631967 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f89ba-a94a-4b1c-a138-94080d9c582b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:34.637690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351c5df-d531-4a28-b0b3-f2695f9e70ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:34.638752 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351c5df-d531-4a28-b0b3-f2695f9e70ad None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:34.639971 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351c5df-d531-4a28-b0b3-f2695f9e70ad None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:34.640417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351c5df-d531-4a28-b0b3-f2695f9e70ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:34.654192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1351c5df-d531-4a28-b0b3-f2695f9e70ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:34.659127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-1351c5df-d531-4a28-b0b3-f2695f9e70ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 23:41:34.659127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:34.659127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:34.659127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:34.659127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:34.659127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:34.659127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:34.659127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:34.659127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:34.659127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:34.659127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:34.659127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:34.659127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:34.659127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:34.659127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:34.659127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:34.660031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:34.660031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:34.660031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:34.660031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:34.660031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:34.660031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:34.660031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:34.660031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:34.660031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:34.660031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:34.660031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:34.660031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:34.660031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:34.660031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:34.660031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:34.660031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:34.660968 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:34.660968 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:34.660968 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:34.660968 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:34.660968 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:34.660968 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:34.660968 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:34.660968 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:34.660968 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:34.660968 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:34.660968 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 23:41:34.660968 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:41:34.660968 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 97/188] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Sep 10 23:41:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:34.661843 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e91e45-5af1-4f30-a8e7-274584f76d5a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:34.662052 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e91e45-5af1-4f30-a8e7-274584f76d5a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:34.662413 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e91e45-5af1-4f30-a8e7-274584f76d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:34.687425 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-51e91e45-5af1-4f30-a8e7-274584f76d5a None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:34.688631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-51e91e45-5af1-4f30-a8e7-274584f76d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:34.725595 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:34.749639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f89ba-a94a-4b1c-a138-94080d9c582b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:34.760239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 98/189] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:34.770874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e91e45-5af1-4f30-a8e7-274584f76d5a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:34.785211 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48602a2e-7df1-4ef7-b852-ec46508bbfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:34.785818 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48602a2e-7df1-4ef7-b852-ec46508bbfc9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:34.786097 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48602a2e-7df1-4ef7-b852-ec46508bbfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:34.786375 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48602a2e-7df1-4ef7-b852-ec46508bbfc9 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:34.787076 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48602a2e-7df1-4ef7-b852-ec46508bbfc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:34.787381 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48602a2e-7df1-4ef7-b852-ec46508bbfc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:34.788762 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48602a2e-7df1-4ef7-b852-ec46508bbfc9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:34.796819 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 92/190] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:34.797615 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48602a2e-7df1-4ef7-b852-ec46508bbfc9 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:34.802191 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:34.883633 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e91e45-5af1-4f30-a8e7-274584f76d5a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:34.905241 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48602a2e-7df1-4ef7-b852-ec46508bbfc9 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:34.918181 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06e698-b056-4a52-95ed-473d07f53754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:34.918829 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06e698-b056-4a52-95ed-473d07f53754 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:34.919104 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06e698-b056-4a52-95ed-473d07f53754 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:34.919936 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06e698-b056-4a52-95ed-473d07f53754 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:34.920661 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a06e698-b056-4a52-95ed-473d07f53754 None admin] RBAC: Adding query filter params (role.id=ff2ed192587843e5b10b5a35949e4c5f, scope.project.id=59ae2524fa884ab7a9682a93d368b500, user.id=067af36c9f9b401d9d35810237e9ba2e) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:34.921118 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a06e698-b056-4a52-95ed-473d07f53754 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:34.922492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a06e698-b056-4a52-95ed-473d07f53754 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:34.928370 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 99/191] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:41:34 2022] GET /identity/v3/role_assignments?user.id=067af36c9f9b401d9d35810237e9ba2e&scope.project.id=59ae2524fa884ab7a9682a93d368b500&role.id=ff2ed192587843e5b10b5a35949e4c5f => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:34.930077 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06e698-b056-4a52-95ed-473d07f53754 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:34.954235 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89b012-a488-44be-8789-bd11f03f41cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:34.954897 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89b012-a488-44be-8789-bd11f03f41cd None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:34.955173 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89b012-a488-44be-8789-bd11f03f41cd None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:34.955443 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89b012-a488-44be-8789-bd11f03f41cd None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:34.956561 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f89b012-a488-44be-8789-bd11f03f41cd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:34.956866 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f89b012-a488-44be-8789-bd11f03f41cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:34.958313 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f89b012-a488-44be-8789-bd11f03f41cd None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:34.968295 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:34.972206 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 93/192] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:41:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:34.973125 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89b012-a488-44be-8789-bd11f03f41cd None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:35.057966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:35.073546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06e698-b056-4a52-95ed-473d07f53754 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:35.100654 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89b012-a488-44be-8789-bd11f03f41cd None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:35.114177 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203e59e-5e77-4d08-b1d2-a7375770feca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:35.115202 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203e59e-5e77-4d08-b1d2-a7375770feca None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:35.115873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203e59e-5e77-4d08-b1d2-a7375770feca None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:35.116695 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203e59e-5e77-4d08-b1d2-a7375770feca None admin] PATH_INFO: `/v3/domains/default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:35.118336 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1203e59e-5e77-4d08-b1d2-a7375770feca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:35.119821 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1203e59e-5e77-4d08-b1d2-a7375770feca None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:35.126559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 100/193] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:41:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:35.126559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203e59e-5e77-4d08-b1d2-a7375770feca None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:35.144473 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a360c8-7ec6-4d7e-959f-f7562d1c1d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:35.145346 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a360c8-7ec6-4d7e-959f-f7562d1c1d58 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:35.145777 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a360c8-7ec6-4d7e-959f-f7562d1c1d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:35.146049 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a360c8-7ec6-4d7e-959f-f7562d1c1d58 None admin] PATH_INFO: `/v3/users/61fe9e83ae764584964d9139676ed85e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:35.163770 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a360c8-7ec6-4d7e-959f-f7562d1c1d58 None admin] RBAC: Authorizing `identity:get_user(user_id=61fe9e83ae764584964d9139676ed85e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:35.165250 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a360c8-7ec6-4d7e-959f-f7562d1c1d58 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:35.167128 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 94/194] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:34 2022] GET /identity/v3/users/61fe9e83ae764584964d9139676ed85e => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:35.167825 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a360c8-7ec6-4d7e-959f-f7562d1c1d58 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:35.235538 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:35.243270 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203e59e-5e77-4d08-b1d2-a7375770feca None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:35.275783 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a360c8-7ec6-4d7e-959f-f7562d1c1d58 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:35.277002 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d990fa-81ce-4429-b226-2a4eae28cd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:35.277625 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d990fa-81ce-4429-b226-2a4eae28cd4b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:35.277900 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d990fa-81ce-4429-b226-2a4eae28cd4b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:35.278172 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d990fa-81ce-4429-b226-2a4eae28cd4b None admin] PATH_INFO: `/v3/users/61fe9e83ae764584964d9139676ed85e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:35.279836 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d990fa-81ce-4429-b226-2a4eae28cd4b None admin] RBAC: Authorizing `identity:get_user(user_id=61fe9e83ae764584964d9139676ed85e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:35.281281 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d990fa-81ce-4429-b226-2a4eae28cd4b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:35.282873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 101/195] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:35 2022] GET /identity/v3/users/61fe9e83ae764584964d9139676ed85e => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:35.283611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d990fa-81ce-4429-b226-2a4eae28cd4b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:35.311210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:35.312041 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80996ee3-5d1b-4071-b136-c3ee18c83b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:35.312675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80996ee3-5d1b-4071-b136-c3ee18c83b7a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:35.312954 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80996ee3-5d1b-4071-b136-c3ee18c83b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:35.313226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80996ee3-5d1b-4071-b136-c3ee18c83b7a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:35.323301 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80996ee3-5d1b-4071-b136-c3ee18c83b7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:35.328626 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-80996ee3-5d1b-4071-b136-c3ee18c83b7a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 23:41:35.328626 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:35.328626 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:35.328626 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:35.328626 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:35.328626 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:35.328626 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:35.328626 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:35.328626 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:35.328626 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:35.328626 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:35.328626 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:35.328626 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:35.328626 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:35.328626 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:35.328626 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:35.332552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:35.332552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:35.332552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:35.332552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:35.332552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:35.332552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:35.332552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:35.332552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:35.332552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:35.332552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:35.332552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:35.332552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:35.332552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:35.332552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:35.332552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:35.332552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:35.333330 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:35.333330 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:35.333330 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:35.333330 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:35.333330 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:35.333330 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:35.333330 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:35.333330 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:35.333330 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:35.333330 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:35.333330 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 23:41:35.333330 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:41:35.333330 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 95/196] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:35.333330 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80996ee3-5d1b-4071-b136-c3ee18c83b7a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:35.386089 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d990fa-81ce-4429-b226-2a4eae28cd4b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:35.418006 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c9cfa-35f2-45e4-9fde-d32e67566d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:35.418595 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c9cfa-35f2-45e4-9fde-d32e67566d5b None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:35.418892 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c9cfa-35f2-45e4-9fde-d32e67566d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:35.419157 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c9cfa-35f2-45e4-9fde-d32e67566d5b None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:35.419873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c9cfa-35f2-45e4-9fde-d32e67566d5b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:35.421307 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c9cfa-35f2-45e4-9fde-d32e67566d5b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:35.428827 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80996ee3-5d1b-4071-b136-c3ee18c83b7a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:35.447500 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:35.464640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef9bdc-fcc5-4298-a27a-e963401aef2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:35.465232 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef9bdc-fcc5-4298-a27a-e963401aef2b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:35.465507 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef9bdc-fcc5-4298-a27a-e963401aef2b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:35.465777 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef9bdc-fcc5-4298-a27a-e963401aef2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:35.522416 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 102/197] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Sep 10 23:41:35 2022] POST /identity/v3/users => generated 297 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:41:35.532185 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef9bdc-fcc5-4298-a27a-e963401aef2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:35.533688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef9bdc-fcc5-4298-a27a-e963401aef2b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:35.535808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c9cfa-35f2-45e4-9fde-d32e67566d5b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:35.630715 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:35.652247 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c9cfa-35f2-45e4-9fde-d32e67566d5b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:35.657074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 96/198] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:41:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:41:35.657963 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef9bdc-fcc5-4298-a27a-e963401aef2b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:35.682912 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:35.692099 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9719143-f9d8-4dd2-b5ee-8249679c9c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:35.692723 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9719143-f9d8-4dd2-b5ee-8249679c9c04 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:35.693000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9719143-f9d8-4dd2-b5ee-8249679c9c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:35.693329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9719143-f9d8-4dd2-b5ee-8249679c9c04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:35.782170 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef9bdc-fcc5-4298-a27a-e963401aef2b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:35.782884 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9719143-f9d8-4dd2-b5ee-8249679c9c04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:35.784780 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9719143-f9d8-4dd2-b5ee-8249679c9c04 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:35.829559 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665b986e-415f-4660-8533-7160bdf213f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:35.830175 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b986e-415f-4660-8533-7160bdf213f2 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:35.830452 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b986e-415f-4660-8533-7160bdf213f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:35.830873 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b986e-415f-4660-8533-7160bdf213f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:35.831553 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665b986e-415f-4660-8533-7160bdf213f2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:35.831919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665b986e-415f-4660-8533-7160bdf213f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:35.833476 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665b986e-415f-4660-8533-7160bdf213f2 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:35.843332 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 97/199] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:41:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:35.844259 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665b986e-415f-4660-8533-7160bdf213f2 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:35.884446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:35.908642 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 103/200] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:41:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:41:35.909297 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9719143-f9d8-4dd2-b5ee-8249679c9c04 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:35.942832 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:35.966887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665b986e-415f-4660-8533-7160bdf213f2 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:36.014069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02487e4a-287e-4382-9bad-5245b3f7df2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:36.014741 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02487e4a-287e-4382-9bad-5245b3f7df2e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:36.015062 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02487e4a-287e-4382-9bad-5245b3f7df2e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:36.015877 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02487e4a-287e-4382-9bad-5245b3f7df2e None admin] PATH_INFO: `/v3/projects/15d53acb4e8c43348673ba66109e0371` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:36.017201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02487e4a-287e-4382-9bad-5245b3f7df2e None admin] RBAC: Authorizing `identity:get_project(project_id=15d53acb4e8c43348673ba66109e0371)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:36.019062 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02487e4a-287e-4382-9bad-5245b3f7df2e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:36.021442 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 98/201] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:35 2022] GET /identity/v3/projects/15d53acb4e8c43348673ba66109e0371 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:36.026756 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02487e4a-287e-4382-9bad-5245b3f7df2e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:36.035023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9719143-f9d8-4dd2-b5ee-8249679c9c04 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:36.072195 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2908f-856a-442b-8e1c-a56f62020345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:36.073316 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2908f-856a-442b-8e1c-a56f62020345 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:36.073924 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2908f-856a-442b-8e1c-a56f62020345 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:36.074391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2908f-856a-442b-8e1c-a56f62020345 None admin] PATH_INFO: `/v3/users/9c30a93a9eea4ac188b9d8e577b934f8` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:36.076378 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2908f-856a-442b-8e1c-a56f62020345 None admin] RBAC: Authorizing `identity:get_user(user_id=9c30a93a9eea4ac188b9d8e577b934f8)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:36.078028 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2908f-856a-442b-8e1c-a56f62020345 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:36.079955 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 104/202] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:35 2022] GET /identity/v3/users/9c30a93a9eea4ac188b9d8e577b934f8 => generated 289 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:36.081006 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2908f-856a-442b-8e1c-a56f62020345 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:36.145805 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:36.149793 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02487e4a-287e-4382-9bad-5245b3f7df2e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:36.196658 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab50459-bd4c-45da-a15e-e69776af2ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:36.197219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab50459-bd4c-45da-a15e-e69776af2ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:36.197500 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab50459-bd4c-45da-a15e-e69776af2ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:36.197774 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab50459-bd4c-45da-a15e-e69776af2ba7 None admin] PATH_INFO: `/v3/projects/15d53acb4e8c43348673ba66109e0371` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:36.199180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab50459-bd4c-45da-a15e-e69776af2ba7 None admin] RBAC: Authorizing `identity:get_project(project_id=15d53acb4e8c43348673ba66109e0371)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:36.202023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab50459-bd4c-45da-a15e-e69776af2ba7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:36.204285 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 99/203] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:36 2022] GET /identity/v3/projects/15d53acb4e8c43348673ba66109e0371 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:36.204488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab50459-bd4c-45da-a15e-e69776af2ba7 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:36.208052 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2908f-856a-442b-8e1c-a56f62020345 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:36.229125 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:36.255084 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b99d1f-0eaf-402b-9674-287e9f6d68fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:36.256194 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b99d1f-0eaf-402b-9674-287e9f6d68fe None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:36.256674 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b99d1f-0eaf-402b-9674-287e9f6d68fe None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:36.257162 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b99d1f-0eaf-402b-9674-287e9f6d68fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:36.258129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b99d1f-0eaf-402b-9674-287e9f6d68fe None admin] RBAC: Adding query filter params (role.id=9dce48a2b53c482d931dcd064b07744e, scope.project.id=15d53acb4e8c43348673ba66109e0371, user.id=61fe9e83ae764584964d9139676ed85e) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:36.258634 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b99d1f-0eaf-402b-9674-287e9f6d68fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:36.260313 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b99d1f-0eaf-402b-9674-287e9f6d68fe None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:36.266803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 105/204] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:41:36 2022] GET /identity/v3/role_assignments?user.id=61fe9e83ae764584964d9139676ed85e&scope.project.id=15d53acb4e8c43348673ba66109e0371&role.id=9dce48a2b53c482d931dcd064b07744e => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:36.273301 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b99d1f-0eaf-402b-9674-287e9f6d68fe None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:36.330816 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab50459-bd4c-45da-a15e-e69776af2ba7 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:36.372440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf560a16-329d-414b-bf0a-cae66f8c9d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:36.373419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf560a16-329d-414b-bf0a-cae66f8c9d02 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:36.373873 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf560a16-329d-414b-bf0a-cae66f8c9d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:36.374235 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf560a16-329d-414b-bf0a-cae66f8c9d02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:36.462924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf560a16-329d-414b-bf0a-cae66f8c9d02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:36.465035 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf560a16-329d-414b-bf0a-cae66f8c9d02 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:36.494121 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:36.506363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b99d1f-0eaf-402b-9674-287e9f6d68fe None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:36.543936 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:36.587277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93fff0-775f-4589-91e9-53c267adbb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:36.588252 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93fff0-775f-4589-91e9-53c267adbb5f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:36.588731 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93fff0-775f-4589-91e9-53c267adbb5f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:36.589227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93fff0-775f-4589-91e9-53c267adbb5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:36.590270 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da93fff0-775f-4589-91e9-53c267adbb5f None admin] RBAC: Adding query filter params (role.id=ff2ed192587843e5b10b5a35949e4c5f, scope.project.id=15d53acb4e8c43348673ba66109e0371, user.id=61fe9e83ae764584964d9139676ed85e) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:36.590836 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da93fff0-775f-4589-91e9-53c267adbb5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:36.592811 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da93fff0-775f-4589-91e9-53c267adbb5f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:36.599638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 106/205] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:41:36 2022] GET /identity/v3/role_assignments?user.id=61fe9e83ae764584964d9139676ed85e&scope.project.id=15d53acb4e8c43348673ba66109e0371&role.id=ff2ed192587843e5b10b5a35949e4c5f => generated 259 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:36.634257 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 100/206] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:41:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:41:36.641016 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93fff0-775f-4589-91e9-53c267adbb5f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:36.753099 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93fff0-775f-4589-91e9-53c267adbb5f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:36.789772 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9eed36-6557-485d-b397-700e62ddc213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:36.790408 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9eed36-6557-485d-b397-700e62ddc213 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:36.790704 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9eed36-6557-485d-b397-700e62ddc213 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:36.790985 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9eed36-6557-485d-b397-700e62ddc213 None admin] PATH_INFO: `/v3/projects/59ae2524fa884ab7a9682a93d368b500` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:36.792109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9eed36-6557-485d-b397-700e62ddc213 None admin] RBAC: Authorizing `identity:get_project(project_id=59ae2524fa884ab7a9682a93d368b500)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:36.793559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9eed36-6557-485d-b397-700e62ddc213 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:36.796227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 107/207] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:36 2022] GET /identity/v3/projects/59ae2524fa884ab7a9682a93d368b500 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:36.802553 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf560a16-329d-414b-bf0a-cae66f8c9d02 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:36.913336 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf560a16-329d-414b-bf0a-cae66f8c9d02 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:36.931827 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:36.952146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccddcbee-a490-42cc-8a15-07813d222b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:36.952754 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccddcbee-a490-42cc-8a15-07813d222b2d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:36.953031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccddcbee-a490-42cc-8a15-07813d222b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:36.953304 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccddcbee-a490-42cc-8a15-07813d222b2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:36.954033 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccddcbee-a490-42cc-8a15-07813d222b2d None admin] RBAC: Adding query filter params (role.id=900ce80935da4fa59be9334c5c8e0083, scope.project.id=59ae2524fa884ab7a9682a93d368b500, user.id=9c30a93a9eea4ac188b9d8e577b934f8) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:36.954336 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccddcbee-a490-42cc-8a15-07813d222b2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:36.955812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccddcbee-a490-42cc-8a15-07813d222b2d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:36.962004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 101/208] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:41:36 2022] GET /identity/v3/role_assignments?user.id=9c30a93a9eea4ac188b9d8e577b934f8&scope.project.id=59ae2524fa884ab7a9682a93d368b500&role.id=900ce80935da4fa59be9334c5c8e0083 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:38.082476 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cd26b-c484-4fdb-a709-6a208159bb1c None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:38.083295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cd26b-c484-4fdb-a709-6a208159bb1c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:38.083295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cd26b-c484-4fdb-a709-6a208159bb1c None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:38.088999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 108/209] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:38.095538 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ddc58-6c2c-4b2e-b164-bcaa89575881 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:38.095538 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ddc58-6c2c-4b2e-b164-bcaa89575881 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:38.095538 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ddc58-6c2c-4b2e-b164-bcaa89575881 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:38.096102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60220cd-a90b-4324-864f-28e04c79adba None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:38.096102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60220cd-a90b-4324-864f-28e04c79adba None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:38.096102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60220cd-a90b-4324-864f-28e04c79adba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:38.096299 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 102/210] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:38.101398 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96509f-fe13-416a-92bf-b91be4076a55 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:38.101885 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96509f-fe13-416a-92bf-b91be4076a55 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:38.102256 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96509f-fe13-416a-92bf-b91be4076a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:38.115300 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-e60220cd-a90b-4324-864f-28e04c79adba None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:38.117105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e60220cd-a90b-4324-864f-28e04c79adba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:38.121700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0f96509f-fe13-416a-92bf-b91be4076a55 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:38.123009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0f96509f-fe13-416a-92bf-b91be4076a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:38.168771 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:38.201366 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 109/211] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 23:41:38.207895 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 103/212] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:38.471288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60220cd-a90b-4324-864f-28e04c79adba None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:38.504449 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f96509f-fe13-416a-92bf-b91be4076a55 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:38.588350 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60220cd-a90b-4324-864f-28e04c79adba None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:38.613417 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f96509f-fe13-416a-92bf-b91be4076a55 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:38.625234 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:38.628840 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecab995-1772-4adf-aa6c-0e37bafe2255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:38.630017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecab995-1772-4adf-aa6c-0e37bafe2255 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:38.630432 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecab995-1772-4adf-aa6c-0e37bafe2255 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:38.631142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecab995-1772-4adf-aa6c-0e37bafe2255 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:38.632405 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecab995-1772-4adf-aa6c-0e37bafe2255 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:38.633982 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecab995-1772-4adf-aa6c-0e37bafe2255 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:38.635793 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 110/213] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:41:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:38.643005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecab995-1772-4adf-aa6c-0e37bafe2255 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:38.655010 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2b421-e036-4bd7-a000-1e286a2c1244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:38.656471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2b421-e036-4bd7-a000-1e286a2c1244 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:38.656471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2b421-e036-4bd7-a000-1e286a2c1244 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:38.656471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2b421-e036-4bd7-a000-1e286a2c1244 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:38.668756 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f2b421-e036-4bd7-a000-1e286a2c1244 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:38.674584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-09f2b421-e036-4bd7-a000-1e286a2c1244 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:38.674584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:38.674584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:38.674584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:38.674584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:38.674584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:38.674584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:38.674584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:38.674584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:38.674584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:38.674584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:38.674584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:38.674584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:38.674584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:38.674584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:38.674584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:38.675734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:38.675734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:38.675734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:38.675734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:38.675734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:38.675734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:38.675734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:38.675734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:38.675734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:38.675734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:38.675734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:38.675734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:38.675734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:38.675734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:38.675734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:38.675734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:38.676705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:38.676705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:38.676705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:38.676705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:38.676705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:38.676705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:38.676705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:38.676705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:38.676705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:38.676705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:38.676705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:38.676705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:41:38.683402 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 104/214] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:41:38.685253 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2b421-e036-4bd7-a000-1e286a2c1244 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:38.702090 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:38.772071 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecab995-1772-4adf-aa6c-0e37bafe2255 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:38.795157 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2b421-e036-4bd7-a000-1e286a2c1244 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:38.809444 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd38cce-2b9d-43e6-ab65-a66895396f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:38.810863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd38cce-2b9d-43e6-ab65-a66895396f24 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:38.811120 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd38cce-2b9d-43e6-ab65-a66895396f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:38.811389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd38cce-2b9d-43e6-ab65-a66895396f24 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:38.812044 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd38cce-2b9d-43e6-ab65-a66895396f24 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:38.813484 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd38cce-2b9d-43e6-ab65-a66895396f24 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:38.835755 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e761f4e8-2338-4f43-b387-3df24527ae96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:38.836378 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761f4e8-2338-4f43-b387-3df24527ae96 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:38.840966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761f4e8-2338-4f43-b387-3df24527ae96 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:38.840966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761f4e8-2338-4f43-b387-3df24527ae96 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:38.840966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e761f4e8-2338-4f43-b387-3df24527ae96 None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:38.840966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e761f4e8-2338-4f43-b387-3df24527ae96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:38.840966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e761f4e8-2338-4f43-b387-3df24527ae96 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:38.847414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 105/215] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:41:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:38.848535 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9ab20-bea4-41b2-bef6-12b16ef3aa6d None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:38.848809 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9ab20-bea4-41b2-bef6-12b16ef3aa6d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:38.849073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9ab20-bea4-41b2-bef6-12b16ef3aa6d None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:38.850405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 106/216] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:38.855028 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9ab20-bea4-41b2-bef6-12b16ef3aa6d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:38.886653 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:38.937800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 111/217] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Sep 10 23:41:38 2022] POST /identity/v3/users => generated 297 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:41:38.939342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f503f-aad8-4a5e-848e-a3f6522eb61e None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:38.939804 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f503f-aad8-4a5e-848e-a3f6522eb61e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:38.940020 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f503f-aad8-4a5e-848e-a3f6522eb61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:38.963056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-1d0f503f-aad8-4a5e-848e-a3f6522eb61e None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:38.964379 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1d0f503f-aad8-4a5e-848e-a3f6522eb61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:38.981162 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9ab20-bea4-41b2-bef6-12b16ef3aa6d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:39.030173 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a052b3-042d-4b56-85ef-e1cf3f1284f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:39.031023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a052b3-042d-4b56-85ef-e1cf3f1284f5 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:39.031530 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a052b3-042d-4b56-85ef-e1cf3f1284f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:39.032303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a052b3-042d-4b56-85ef-e1cf3f1284f5 None admin] PATH_INFO: `/v3/users/d410102c90b843c691b4401ffb232f68` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:39.048888 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a052b3-042d-4b56-85ef-e1cf3f1284f5 None admin] RBAC: Authorizing `identity:get_user(user_id=d410102c90b843c691b4401ffb232f68)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:39.051195 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a052b3-042d-4b56-85ef-e1cf3f1284f5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:39.054698 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 107/218] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:38 2022] GET /identity/v3/users/d410102c90b843c691b4401ffb232f68 => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:41:39.060668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a052b3-042d-4b56-85ef-e1cf3f1284f5 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:39.074907 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 112/219] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:39.164548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:39.171058 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918f7b9-edb8-403b-9c50-3eddcd115be1 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:39.171500 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918f7b9-edb8-403b-9c50-3eddcd115be1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:39.171600 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918f7b9-edb8-403b-9c50-3eddcd115be1 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:39.173019 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 113/220] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:41:39.175594 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a052b3-042d-4b56-85ef-e1cf3f1284f5 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:39.179739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a237528-0e85-4050-9e65-c5a70c10f029 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:39.179739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a237528-0e85-4050-9e65-c5a70c10f029 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:39.179739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a237528-0e85-4050-9e65-c5a70c10f029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:39.200199 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-0a237528-0e85-4050-9e65-c5a70c10f029 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:39.201743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0a237528-0e85-4050-9e65-c5a70c10f029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:39.218847 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1faf3-222b-4bc8-970b-519a27eb71b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:39.219683 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1faf3-222b-4bc8-970b-519a27eb71b5 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:39.220098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1faf3-222b-4bc8-970b-519a27eb71b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:39.220517 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1faf3-222b-4bc8-970b-519a27eb71b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:39.238775 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:39.293117 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 114/221] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:39.314188 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1faf3-222b-4bc8-970b-519a27eb71b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:39.315775 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1faf3-222b-4bc8-970b-519a27eb71b5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:39.350766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a237528-0e85-4050-9e65-c5a70c10f029 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:39.427813 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:39.443523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 108/222] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:41:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:41:39.451296 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1faf3-222b-4bc8-970b-519a27eb71b5 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:39.493681 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a237528-0e85-4050-9e65-c5a70c10f029 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:39.549517 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe97474-d6d3-45d7-8da7-89bc6d8498e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:39.550533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe97474-d6d3-45d7-8da7-89bc6d8498e4 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:39.552244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe97474-d6d3-45d7-8da7-89bc6d8498e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:39.552708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe97474-d6d3-45d7-8da7-89bc6d8498e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:39.566504 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe97474-d6d3-45d7-8da7-89bc6d8498e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:39.571649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-8fe97474-d6d3-45d7-8da7-89bc6d8498e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:41:39.571649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:39.571649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:39.571649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:39.571649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:39.571649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:39.571649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:39.571649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:39.571649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:39.571649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:39.571649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:39.571649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:39.571649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:39.571649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:39.571649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:39.571649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:39.572492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:39.572492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:39.572492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:39.572492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:39.572492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:39.572492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:39.572492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:39.572492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:39.572492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:39.572492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:39.572492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:39.572492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:39.572492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:39.572492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:39.572492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:39.572492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:39.573274 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:39.573274 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:39.573274 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:39.573274 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:39.573274 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:39.573274 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:39.573274 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:39.573274 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:39.573274 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:39.573274 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:39.573274 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:41:39.573274 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:41:39.576187 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 115/223] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:41:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:41:39.582006 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe97474-d6d3-45d7-8da7-89bc6d8498e4 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:39.607659 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1faf3-222b-4bc8-970b-519a27eb71b5 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:39.626470 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:39.650053 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e839fc3c-e2a2-434c-a0f5-06dda1fc2a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:39.650702 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839fc3c-e2a2-434c-a0f5-06dda1fc2a42 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:39.651132 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839fc3c-e2a2-434c-a0f5-06dda1fc2a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:39.651415 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839fc3c-e2a2-434c-a0f5-06dda1fc2a42 None admin] PATH_INFO: `/v3/projects/15d53acb4e8c43348673ba66109e0371` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:39.652880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e839fc3c-e2a2-434c-a0f5-06dda1fc2a42 None admin] RBAC: Authorizing `identity:get_project(project_id=15d53acb4e8c43348673ba66109e0371)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:39.656303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e839fc3c-e2a2-434c-a0f5-06dda1fc2a42 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:39.657414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 109/224] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:39 2022] GET /identity/v3/projects/15d53acb4e8c43348673ba66109e0371 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:41:39.658273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e839fc3c-e2a2-434c-a0f5-06dda1fc2a42 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:39.701480 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe97474-d6d3-45d7-8da7-89bc6d8498e4 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:39.736859 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:39.740941 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6efa3-e57f-4d73-b901-7f8cb9fd714f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:39.741545 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6efa3-e57f-4d73-b901-7f8cb9fd714f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:39.741819 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6efa3-e57f-4d73-b901-7f8cb9fd714f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:39.742085 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6efa3-e57f-4d73-b901-7f8cb9fd714f None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:39.742771 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b6efa3-e57f-4d73-b901-7f8cb9fd714f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:39.743071 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b6efa3-e57f-4d73-b901-7f8cb9fd714f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:39.744508 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b6efa3-e57f-4d73-b901-7f8cb9fd714f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:39.753660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 116/225] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:41:39.754487 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6efa3-e57f-4d73-b901-7f8cb9fd714f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:39.788548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e839fc3c-e2a2-434c-a0f5-06dda1fc2a42 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:39.827307 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b08e8d-7fb0-404a-ad53-23961ecd9ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:39.828356 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b08e8d-7fb0-404a-ad53-23961ecd9ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:39.828824 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b08e8d-7fb0-404a-ad53-23961ecd9ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:39.829370 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b08e8d-7fb0-404a-ad53-23961ecd9ad4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:39.840601 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b08e8d-7fb0-404a-ad53-23961ecd9ad4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:39.845663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:39.848802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-34b08e8d-7fb0-404a-ad53-23961ecd9ad4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 23:41:39.848802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:39.848802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:39.848802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:39.848802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:39.848802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:39.848802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:39.848802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:39.848802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:39.848802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:39.848802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:39.848802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:39.848802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:39.848802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:39.848802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:39.848802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:39.849639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:39.849639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:39.849639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:39.849639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:39.849639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:39.849639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:39.849639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:39.849639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:39.849639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:39.849639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:39.849639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:39.849639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:39.849639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:39.849639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:39.849639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:39.849639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:39.850448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:39.850448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:39.850448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:39.850448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:39.850448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:39.850448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:39.850448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:39.850448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:39.850448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:39.850448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:39.850448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 23:41:39.850448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:41:39.852336 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 110/226] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Sep 10 23:41:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:39.853558 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b08e8d-7fb0-404a-ad53-23961ecd9ad4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:39.865515 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6efa3-e57f-4d73-b901-7f8cb9fd714f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:39.903392 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b3f3f-8b5d-427f-a66d-acc51c0d692f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:39.904823 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b3f3f-8b5d-427f-a66d-acc51c0d692f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:39.905223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b3f3f-8b5d-427f-a66d-acc51c0d692f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:39.905451 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b3f3f-8b5d-427f-a66d-acc51c0d692f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:39.906180 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b3f3f-8b5d-427f-a66d-acc51c0d692f None admin] RBAC: Adding query filter params (role.id=824f5fe1f02b447382190a8c236c27db, scope.project.id=15d53acb4e8c43348673ba66109e0371, user.id=d410102c90b843c691b4401ffb232f68) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:39.906486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b3f3f-8b5d-427f-a66d-acc51c0d692f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:39.908119 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b3f3f-8b5d-427f-a66d-acc51c0d692f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:39.914713 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 117/227] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:41:39 2022] GET /identity/v3/role_assignments?user.id=d410102c90b843c691b4401ffb232f68&scope.project.id=15d53acb4e8c43348673ba66109e0371&role.id=824f5fe1f02b447382190a8c236c27db => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:39.914948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b3f3f-8b5d-427f-a66d-acc51c0d692f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:40.039937 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b08e8d-7fb0-404a-ad53-23961ecd9ad4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:40.082718 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:40.127171 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae805c7-049c-4308-aafe-ba37088daba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:40.127809 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae805c7-049c-4308-aafe-ba37088daba7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:40.128085 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae805c7-049c-4308-aafe-ba37088daba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:40.128354 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae805c7-049c-4308-aafe-ba37088daba7 None admin] PATH_INFO: `/v3/users/61fe9e83ae764584964d9139676ed85e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:40.130131 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae805c7-049c-4308-aafe-ba37088daba7 None admin] RBAC: Authorizing `identity:get_user(user_id=61fe9e83ae764584964d9139676ed85e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:40.133235 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae805c7-049c-4308-aafe-ba37088daba7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:40.133235 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 111/228] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:39 2022] GET /identity/v3/users/61fe9e83ae764584964d9139676ed85e => generated 283 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:40.134354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b3f3f-8b5d-427f-a66d-acc51c0d692f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:40.143704 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae805c7-049c-4308-aafe-ba37088daba7 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:40.182982 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d6640-519c-4167-b8bb-162b41718c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:40.184293 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d6640-519c-4167-b8bb-162b41718c91 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:40.184791 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d6640-519c-4167-b8bb-162b41718c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:40.185896 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d6640-519c-4167-b8bb-162b41718c91 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:40.186975 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d6640-519c-4167-b8bb-162b41718c91 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:40.187470 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d6640-519c-4167-b8bb-162b41718c91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:40.189081 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d6640-519c-4167-b8bb-162b41718c91 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:40.198080 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 118/229] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:41:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:40.207081 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d6640-519c-4167-b8bb-162b41718c91 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:40.217939 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:40.273814 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae805c7-049c-4308-aafe-ba37088daba7 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:40.310106 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011a5ec-9624-44a8-b12e-87211d16ce01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:40.311118 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011a5ec-9624-44a8-b12e-87211d16ce01 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:40.312344 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011a5ec-9624-44a8-b12e-87211d16ce01 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:40.312833 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011a5ec-9624-44a8-b12e-87211d16ce01 None admin] PATH_INFO: `/v3/projects/15d53acb4e8c43348673ba66109e0371` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:40.314175 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f011a5ec-9624-44a8-b12e-87211d16ce01 None admin] RBAC: Authorizing `identity:get_project(project_id=15d53acb4e8c43348673ba66109e0371)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:40.315875 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f011a5ec-9624-44a8-b12e-87211d16ce01 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:40.317730 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 112/230] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:40 2022] GET /identity/v3/projects/15d53acb4e8c43348673ba66109e0371 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:40.326622 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011a5ec-9624-44a8-b12e-87211d16ce01 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:40.384226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:40.429268 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d6640-519c-4167-b8bb-162b41718c91 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:40.440860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011a5ec-9624-44a8-b12e-87211d16ce01 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:40.477428 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772e777-e215-4e3b-8fa6-433442a6614c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:40.478500 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772e777-e215-4e3b-8fa6-433442a6614c None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:40.479100 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772e777-e215-4e3b-8fa6-433442a6614c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:40.479565 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772e777-e215-4e3b-8fa6-433442a6614c None admin] PATH_INFO: `/v3/projects/15d53acb4e8c43348673ba66109e0371/users/61fe9e83ae764584964d9139676ed85e/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:40.481805 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f772e777-e215-4e3b-8fa6-433442a6614c None admin] RBAC: Authorizing `identity:create_grant(project_id=15d53acb4e8c43348673ba66109e0371, user_id=61fe9e83ae764584964d9139676ed85e, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:40.483489 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f772e777-e215-4e3b-8fa6-433442a6614c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:40.494883 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1d1d5-0086-4672-a849-36dc825e0f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:40.495869 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1d1d5-0086-4672-a849-36dc825e0f85 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:40.496382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1d1d5-0086-4672-a849-36dc825e0f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:40.496991 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1d1d5-0086-4672-a849-36dc825e0f85 None admin] PATH_INFO: `/v3/users/61fe9e83ae764584964d9139676ed85e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:40.499031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e1d1d5-0086-4672-a849-36dc825e0f85 None admin] RBAC: Authorizing `identity:get_user(user_id=61fe9e83ae764584964d9139676ed85e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:40.500760 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e1d1d5-0086-4672-a849-36dc825e0f85 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:40.514225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 119/231] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:40 2022] GET /identity/v3/users/61fe9e83ae764584964d9139676ed85e => generated 283 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:40.514225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1d1d5-0086-4672-a849-36dc825e0f85 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:40.568371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 113/232] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Sep 10 23:41:40 2022] PUT /identity/v3/projects/15d53acb4e8c43348673ba66109e0371/users/61fe9e83ae764584964d9139676ed85e/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:41:40.600600 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:40.631625 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1d1d5-0086-4672-a849-36dc825e0f85 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:40.687650 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb1cb0-723d-49b4-b061-c23620beda2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:40.688742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb1cb0-723d-49b4-b061-c23620beda2e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:40.689218 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb1cb0-723d-49b4-b061-c23620beda2e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:40.689736 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb1cb0-723d-49b4-b061-c23620beda2e None admin] PATH_INFO: `/v3/projects/15d53acb4e8c43348673ba66109e0371` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:40.691268 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb1cb0-723d-49b4-b061-c23620beda2e None admin] RBAC: Authorizing `identity:get_project(project_id=15d53acb4e8c43348673ba66109e0371)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:40.692977 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb1cb0-723d-49b4-b061-c23620beda2e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:40.694927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 120/233] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:40 2022] GET /identity/v3/projects/15d53acb4e8c43348673ba66109e0371 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:41:40.702856 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772e777-e215-4e3b-8fa6-433442a6614c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:40.818132 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772e777-e215-4e3b-8fa6-433442a6614c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:40.839612 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:40.853668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3149c79a-f9e2-4adc-bd6a-18a919992ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:40.855134 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149c79a-f9e2-4adc-bd6a-18a919992ed9 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:40.855134 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149c79a-f9e2-4adc-bd6a-18a919992ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:40.855134 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149c79a-f9e2-4adc-bd6a-18a919992ed9 None admin] PATH_INFO: `/v3/projects/15d53acb4e8c43348673ba66109e0371/users/61fe9e83ae764584964d9139676ed85e/roles/ff2ed192587843e5b10b5a35949e4c5f` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:40.858063 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3149c79a-f9e2-4adc-bd6a-18a919992ed9 None admin] RBAC: Authorizing `identity:create_grant(project_id=15d53acb4e8c43348673ba66109e0371, user_id=61fe9e83ae764584964d9139676ed85e, role_id=ff2ed192587843e5b10b5a35949e4c5f)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:40.860147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3149c79a-f9e2-4adc-bd6a-18a919992ed9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:40.960824 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 114/234] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Sep 10 23:41:40 2022] PUT /identity/v3/projects/15d53acb4e8c43348673ba66109e0371/users/61fe9e83ae764584964d9139676ed85e/roles/ff2ed192587843e5b10b5a35949e4c5f => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:41:41.640657 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63c43c-dfcf-491c-aee2-0548919aca10 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:41.641142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63c43c-dfcf-491c-aee2-0548919aca10 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:41.641142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63c43c-dfcf-491c-aee2-0548919aca10 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:41.642680 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 121/235] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:41.647970 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65125c7c-144d-4d12-9dad-c80415c76a0e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:41.648364 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65125c7c-144d-4d12-9dad-c80415c76a0e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:41.648504 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65125c7c-144d-4d12-9dad-c80415c76a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:41.674890 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-65125c7c-144d-4d12-9dad-c80415c76a0e None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:41.676074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-65125c7c-144d-4d12-9dad-c80415c76a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:41.762815 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 115/236] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:41.968326 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1427280-6507-4424-8e97-77f966ba9e15 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:41.968800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1427280-6507-4424-8e97-77f966ba9e15 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:41.969279 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1427280-6507-4424-8e97-77f966ba9e15 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:41.976052 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 122/237] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:41.978800 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e4061-40eb-4a86-a877-34861d53eb1f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:41.979295 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e4061-40eb-4a86-a877-34861d53eb1f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:41.979766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e4061-40eb-4a86-a877-34861d53eb1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:42.007623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-707e4061-40eb-4a86-a877-34861d53eb1f None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:42.009150 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-707e4061-40eb-4a86-a877-34861d53eb1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:42.092684 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 116/238] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:42.375618 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1427280-6507-4424-8e97-77f966ba9e15 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:42.552811 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1427280-6507-4424-8e97-77f966ba9e15 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:42.608792 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756690b8-5372-44ab-8f98-9f4bada23436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:42.609695 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756690b8-5372-44ab-8f98-9f4bada23436 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:42.610170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756690b8-5372-44ab-8f98-9f4bada23436 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:42.610609 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756690b8-5372-44ab-8f98-9f4bada23436 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:42.612442 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756690b8-5372-44ab-8f98-9f4bada23436 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:42.615050 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756690b8-5372-44ab-8f98-9f4bada23436 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:42.617725 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 123/239] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:41:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:42.623889 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e4061-40eb-4a86-a877-34861d53eb1f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:42.685221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:42.690282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756690b8-5372-44ab-8f98-9f4bada23436 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:42.723174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:42.790236 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e4061-40eb-4a86-a877-34861d53eb1f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:42.830883 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47349b0-d686-48b1-b049-56b63fb37744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:42.832136 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47349b0-d686-48b1-b049-56b63fb37744 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:42.832663 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47349b0-d686-48b1-b049-56b63fb37744 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:42.833092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47349b0-d686-48b1-b049-56b63fb37744 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:42.833898 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47349b0-d686-48b1-b049-56b63fb37744 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:42.835637 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47349b0-d686-48b1-b049-56b63fb37744 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:42.883583 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756690b8-5372-44ab-8f98-9f4bada23436 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:42.922329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 117/240] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Sep 10 23:41:42 2022] POST /identity/v3/users => generated 293 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 23:41:42.923914 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150d557-addd-4059-a4e3-44d616a0e368 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:42.924204 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150d557-addd-4059-a4e3-44d616a0e368 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:42.924474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150d557-addd-4059-a4e3-44d616a0e368 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:42.925596 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0db7a6-78e5-4a85-b77d-12fdba8c027e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:42.926308 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 118/241] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:42.926710 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0db7a6-78e5-4a85-b77d-12fdba8c027e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:42.936598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0db7a6-78e5-4a85-b77d-12fdba8c027e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:42.936598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0db7a6-78e5-4a85-b77d-12fdba8c027e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:42.936825 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc5624-925f-4fe9-bf3a-f7e79fbf2dd3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:42.936825 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc5624-925f-4fe9-bf3a-f7e79fbf2dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:42.936825 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc5624-925f-4fe9-bf3a-f7e79fbf2dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:42.953740 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0db7a6-78e5-4a85-b77d-12fdba8c027e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:42.958276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a5fc5624-925f-4fe9-bf3a-f7e79fbf2dd3 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:42.959672 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a5fc5624-925f-4fe9-bf3a-f7e79fbf2dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:42.959942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-ea0db7a6-78e5-4a85-b77d-12fdba8c027e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 23:41:42.959942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:42.959942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:42.959942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:42.959942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:42.959942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:42.959942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:42.959942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:42.959942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:42.959942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:42.959942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:42.959942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:42.959942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:42.959942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:42.959942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:42.959942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:42.960981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:42.960981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:42.960981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:42.960981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:42.960981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:42.960981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:42.960981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:42.960981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:42.960981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:42.960981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:42.960981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:42.960981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:42.960981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:42.960981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:42.960981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:42.960981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:42.961748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:42.961748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:42.961748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:42.961748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:42.961748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:42.961748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:42.961748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:42.961748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:42.961748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:42.961748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:42.961748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 23:41:42.961748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:41:42.964147 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 124/242] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:41:42.970937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0db7a6-78e5-4a85-b77d-12fdba8c027e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:43.050216 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:43.064178 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 119/243] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:43.086730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0db7a6-78e5-4a85-b77d-12fdba8c027e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:43.140976 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c521bc-0589-4a3b-aafa-62d81d049fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:43.141504 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c521bc-0589-4a3b-aafa-62d81d049fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:43.141775 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c521bc-0589-4a3b-aafa-62d81d049fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:43.142037 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c521bc-0589-4a3b-aafa-62d81d049fa3 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:43.142801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c521bc-0589-4a3b-aafa-62d81d049fa3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:43.143103 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c521bc-0589-4a3b-aafa-62d81d049fa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:43.144485 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c521bc-0589-4a3b-aafa-62d81d049fa3 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:43.154083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 125/244] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:41:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:43.159838 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc5624-925f-4fe9-bf3a-f7e79fbf2dd3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:43.191801 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:43.291765 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ae995-8ab4-41f8-846d-cf966b628c2a None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:43.292095 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ae995-8ab4-41f8-846d-cf966b628c2a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:43.292315 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ae995-8ab4-41f8-846d-cf966b628c2a None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:43.293656 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 126/245] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:41:43.306422 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbf6b6-3c55-4775-91b5-1fcd55c8f97e None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:43.307003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbf6b6-3c55-4775-91b5-1fcd55c8f97e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:43.307262 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbf6b6-3c55-4775-91b5-1fcd55c8f97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:43.324047 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc5624-925f-4fe9-bf3a-f7e79fbf2dd3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:43.332330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-dedbf6b6-3c55-4775-91b5-1fcd55c8f97e None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:43.333521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-dedbf6b6-3c55-4775-91b5-1fcd55c8f97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:43.374602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684c1ea-1790-4bb2-b7d4-115ed374c28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:43.375799 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684c1ea-1790-4bb2-b7d4-115ed374c28d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:43.376093 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684c1ea-1790-4bb2-b7d4-115ed374c28d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:43.376363 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684c1ea-1790-4bb2-b7d4-115ed374c28d None admin] PATH_INFO: `/v3/users/67b2fb0ba6574c42a17448654d297564` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:43.394857 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684c1ea-1790-4bb2-b7d4-115ed374c28d None admin] RBAC: Authorizing `identity:get_user(user_id=67b2fb0ba6574c42a17448654d297564)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:43.396937 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684c1ea-1790-4bb2-b7d4-115ed374c28d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:43.398778 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 120/246] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:43 2022] GET /identity/v3/users/67b2fb0ba6574c42a17448654d297564 => generated 297 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:43.399443 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684c1ea-1790-4bb2-b7d4-115ed374c28d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:43.430780 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 127/247] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 23:41:43.431888 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbf6b6-3c55-4775-91b5-1fcd55c8f97e None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:43.491569 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:43.505108 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:43.516292 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684c1ea-1790-4bb2-b7d4-115ed374c28d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:43.537800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbf6b6-3c55-4775-91b5-1fcd55c8f97e None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:43.565293 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aad949-978b-43a2-be82-8a91ae5f41f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:43.566163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aad949-978b-43a2-be82-8a91ae5f41f9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:43.566783 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aad949-978b-43a2-be82-8a91ae5f41f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:43.567223 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aad949-978b-43a2-be82-8a91ae5f41f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:43.576000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07446408-fbda-4d00-92f0-73b35ba42c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:43.576697 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07446408-fbda-4d00-92f0-73b35ba42c7f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:43.577074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07446408-fbda-4d00-92f0-73b35ba42c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:43.577442 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07446408-fbda-4d00-92f0-73b35ba42c7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:43.579847 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aad949-978b-43a2-be82-8a91ae5f41f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:43.584958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-90aad949-978b-43a2-be82-8a91ae5f41f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:43.584958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:43.584958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:43.584958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:43.584958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:43.584958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:43.584958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:43.584958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:43.584958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:43.584958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:43.584958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:43.584958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:43.584958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:43.584958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:43.584958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:43.584958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:43.585906 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:43.585906 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:43.585906 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:43.585906 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:43.585906 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:43.585906 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:43.585906 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:43.585906 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:43.585906 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:43.585906 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:43.585906 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:43.585906 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:43.585906 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:43.585906 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:43.585906 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:43.585906 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:43.586691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:43.586691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:43.586691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:43.586691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:43.586691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:43.586691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:43.586691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:43.586691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:43.586691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:43.586691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:43.586691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:43.586691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:41:43.594963 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 121/248] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:43.597722 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aad949-978b-43a2-be82-8a91ae5f41f9 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:43.656304 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07446408-fbda-4d00-92f0-73b35ba42c7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:43.657921 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07446408-fbda-4d00-92f0-73b35ba42c7f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:43.719352 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aad949-978b-43a2-be82-8a91ae5f41f9 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:43.723679 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:43.761277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce8a52-c435-41ca-9fcd-e2bcb23081ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:43.762197 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce8a52-c435-41ca-9fcd-e2bcb23081ee None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:43.762650 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce8a52-c435-41ca-9fcd-e2bcb23081ee None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:43.763903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce8a52-c435-41ca-9fcd-e2bcb23081ee None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:43.764779 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dce8a52-c435-41ca-9fcd-e2bcb23081ee None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:43.765277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dce8a52-c435-41ca-9fcd-e2bcb23081ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:43.766933 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dce8a52-c435-41ca-9fcd-e2bcb23081ee None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:43.769793 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 128/249] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:41:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:41:43.773463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:43.776560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07446408-fbda-4d00-92f0-73b35ba42c7f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:43.777406 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 122/250] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:41:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:43.784375 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce8a52-c435-41ca-9fcd-e2bcb23081ee None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:43.902902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07446408-fbda-4d00-92f0-73b35ba42c7f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:43.914289 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce8a52-c435-41ca-9fcd-e2bcb23081ee None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:43.943705 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1635b93-86c8-4194-826a-878e0a6e27e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:43.944342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1635b93-86c8-4194-826a-878e0a6e27e6 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:43.944613 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1635b93-86c8-4194-826a-878e0a6e27e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:43.944970 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1635b93-86c8-4194-826a-878e0a6e27e6 None admin] PATH_INFO: `/v3/projects/15d53acb4e8c43348673ba66109e0371` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:43.946101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1635b93-86c8-4194-826a-878e0a6e27e6 None admin] RBAC: Authorizing `identity:get_project(project_id=15d53acb4e8c43348673ba66109e0371)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:43.948069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1635b93-86c8-4194-826a-878e0a6e27e6 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:43.949959 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 129/251] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:43 2022] GET /identity/v3/projects/15d53acb4e8c43348673ba66109e0371 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:41:43.951582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1635b93-86c8-4194-826a-878e0a6e27e6 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:43.958278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abe374-4a97-4d5e-a899-4330bec7bb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:43.959941 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abe374-4a97-4d5e-a899-4330bec7bb79 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:43.962916 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abe374-4a97-4d5e-a899-4330bec7bb79 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:43.963419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abe374-4a97-4d5e-a899-4330bec7bb79 None admin] PATH_INFO: `/v3/users/d410102c90b843c691b4401ffb232f68` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:43.965365 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abe374-4a97-4d5e-a899-4330bec7bb79 None admin] RBAC: Authorizing `identity:get_user(user_id=d410102c90b843c691b4401ffb232f68)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:43.967783 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abe374-4a97-4d5e-a899-4330bec7bb79 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:43.970302 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 123/252] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:43 2022] GET /identity/v3/users/d410102c90b843c691b4401ffb232f68 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:43.971541 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abe374-4a97-4d5e-a899-4330bec7bb79 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:43.990032 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:44.059241 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:44.111539 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abe374-4a97-4d5e-a899-4330bec7bb79 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:44.130416 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1635b93-86c8-4194-826a-878e0a6e27e6 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:44.175093 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa3eefb-d2cd-4eae-bd13-340ba88fe799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:44.175705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa3eefb-d2cd-4eae-bd13-340ba88fe799 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:44.176073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa3eefb-d2cd-4eae-bd13-340ba88fe799 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:44.176348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa3eefb-d2cd-4eae-bd13-340ba88fe799 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:44.177073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa3eefb-d2cd-4eae-bd13-340ba88fe799 None admin] RBAC: Adding query filter params (role.id=900ce80935da4fa59be9334c5c8e0083, scope.project.id=15d53acb4e8c43348673ba66109e0371, user.id=67b2fb0ba6574c42a17448654d297564) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:44.177374 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa3eefb-d2cd-4eae-bd13-340ba88fe799 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:44.179105 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa3eefb-d2cd-4eae-bd13-340ba88fe799 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:44.184226 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be038f-1ce9-4b3f-835a-f5799c53eaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:44.185723 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be038f-1ce9-4b3f-835a-f5799c53eaac None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:44.185833 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 124/253] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:41:43 2022] GET /identity/v3/role_assignments?user.id=67b2fb0ba6574c42a17448654d297564&scope.project.id=15d53acb4e8c43348673ba66109e0371&role.id=900ce80935da4fa59be9334c5c8e0083 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:44.186000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be038f-1ce9-4b3f-835a-f5799c53eaac None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:44.186267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be038f-1ce9-4b3f-835a-f5799c53eaac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:44.187042 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa3eefb-d2cd-4eae-bd13-340ba88fe799 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:44.198116 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be038f-1ce9-4b3f-835a-f5799c53eaac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:44.203835 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-05be038f-1ce9-4b3f-835a-f5799c53eaac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:41:44.203835 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:44.203835 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:44.203835 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:44.203835 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:44.203835 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:44.203835 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:44.203835 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:44.203835 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:44.203835 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:44.203835 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:44.203835 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:44.203835 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:44.203835 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:44.203835 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:44.203835 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:44.204708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:44.204708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:44.204708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:44.204708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:44.204708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:44.204708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:44.204708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:44.204708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:44.204708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:44.204708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:44.204708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:44.204708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:44.204708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:44.204708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:44.204708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:44.204708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:44.205495 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:44.205495 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:44.205495 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:44.205495 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:44.205495 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:44.205495 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:44.205495 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:44.205495 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:44.205495 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:44.205495 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:44.205495 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:41:44.205495 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:41:44.205495 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 130/254] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:41:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:44.206847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf90ead-0c6b-4450-9825-6225943e4a96 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:44.206847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf90ead-0c6b-4450-9825-6225943e4a96 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:44.207164 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf90ead-0c6b-4450-9825-6225943e4a96 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:44.208583 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 131/255] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:44.213557 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf90ead-0c6b-4450-9825-6225943e4a96 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:44.369339 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa3eefb-d2cd-4eae-bd13-340ba88fe799 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:44.386705 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:44.406122 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:44.408435 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf90ead-0c6b-4450-9825-6225943e4a96 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:44.417002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c168df-613c-4032-a606-8081a031b36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:44.417966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c168df-613c-4032-a606-8081a031b36f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:44.418246 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c168df-613c-4032-a606-8081a031b36f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:44.418521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c168df-613c-4032-a606-8081a031b36f None admin] PATH_INFO: `/v3/projects/15d53acb4e8c43348673ba66109e0371` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:44.421988 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c168df-613c-4032-a606-8081a031b36f None admin] RBAC: Authorizing `identity:get_project(project_id=15d53acb4e8c43348673ba66109e0371)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:44.424159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c168df-613c-4032-a606-8081a031b36f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:44.426054 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 125/256] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:44 2022] GET /identity/v3/projects/15d53acb4e8c43348673ba66109e0371 => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:44.432721 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e117a-5a5f-478f-b69e-94400d928dee None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:44.436345 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e117a-5a5f-478f-b69e-94400d928dee None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:44.436345 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e117a-5a5f-478f-b69e-94400d928dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:44.457859 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c77e117a-5a5f-478f-b69e-94400d928dee None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:44.458291 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd6e1b-51c0-4a58-8d24-dee9814579a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:44.459178 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c77e117a-5a5f-478f-b69e-94400d928dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:44.459495 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd6e1b-51c0-4a58-8d24-dee9814579a0 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:44.459956 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd6e1b-51c0-4a58-8d24-dee9814579a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:44.460408 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd6e1b-51c0-4a58-8d24-dee9814579a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:44.461479 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd6e1b-51c0-4a58-8d24-dee9814579a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:44.462865 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd6e1b-51c0-4a58-8d24-dee9814579a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:44.464591 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd6e1b-51c0-4a58-8d24-dee9814579a0 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:44.480915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 132/257] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:44.482226 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd6e1b-51c0-4a58-8d24-dee9814579a0 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:44.596192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd6e1b-51c0-4a58-8d24-dee9814579a0 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:44.632908 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96a34d-7771-4d76-b99f-53819f8efab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:44.633907 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96a34d-7771-4d76-b99f-53819f8efab4 None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:44.634375 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96a34d-7771-4d76-b99f-53819f8efab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:44.634863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96a34d-7771-4d76-b99f-53819f8efab4 None admin] PATH_INFO: `/v3/projects/15d53acb4e8c43348673ba66109e0371/users/d410102c90b843c691b4401ffb232f68/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:44.637133 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de96a34d-7771-4d76-b99f-53819f8efab4 None admin] RBAC: Authorizing `identity:create_grant(project_id=15d53acb4e8c43348673ba66109e0371, user_id=d410102c90b843c691b4401ffb232f68, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:44.639771 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de96a34d-7771-4d76-b99f-53819f8efab4 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:44.644225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 126/258] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:44.644930 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e117a-5a5f-478f-b69e-94400d928dee None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:44.720393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 133/259] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Sep 10 23:41:44 2022] PUT /identity/v3/projects/15d53acb4e8c43348673ba66109e0371/users/d410102c90b843c691b4401ffb232f68/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:41:44.765229 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e117a-5a5f-478f-b69e-94400d928dee None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:44.807382 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:44.814679 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd69958e-2c3b-40d7-bb22-1a927b29a031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:44.815618 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69958e-2c3b-40d7-bb22-1a927b29a031 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:44.816053 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69958e-2c3b-40d7-bb22-1a927b29a031 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:44.816476 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69958e-2c3b-40d7-bb22-1a927b29a031 None admin] PATH_INFO: `/v3/users/61fe9e83ae764584964d9139676ed85e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:44.819309 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd69958e-2c3b-40d7-bb22-1a927b29a031 None admin] RBAC: Authorizing `identity:get_user(user_id=61fe9e83ae764584964d9139676ed85e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:44.819804 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd69958e-2c3b-40d7-bb22-1a927b29a031 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:44.821684 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 127/260] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:44 2022] GET /identity/v3/users/61fe9e83ae764584964d9139676ed85e => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:44.828350 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96a34d-7771-4d76-b99f-53819f8efab4 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:44.847339 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:44.961196 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96a34d-7771-4d76-b99f-53819f8efab4 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:45.001344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0a5ae-12a9-4eb9-816c-a5b588c541a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:45.002327 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0a5ae-12a9-4eb9-816c-a5b588c541a8 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:45.002822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0a5ae-12a9-4eb9-816c-a5b588c541a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:45.003283 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0a5ae-12a9-4eb9-816c-a5b588c541a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:45.096828 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e0a5ae-12a9-4eb9-816c-a5b588c541a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:45.098697 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e0a5ae-12a9-4eb9-816c-a5b588c541a8 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:45.120954 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:45.207087 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd69958e-2c3b-40d7-bb22-1a927b29a031 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:45.274080 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 134/261] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:41:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:41:45.292069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0a5ae-12a9-4eb9-816c-a5b588c541a8 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:45.431194 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0a5ae-12a9-4eb9-816c-a5b588c541a8 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:45.446435 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd69958e-2c3b-40d7-bb22-1a927b29a031 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:45.451632 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:45.489669 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe7e5e-eef1-47a6-940c-521473cea073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:45.490356 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe7e5e-eef1-47a6-940c-521473cea073 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:45.490642 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe7e5e-eef1-47a6-940c-521473cea073 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:45.490951 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe7e5e-eef1-47a6-940c-521473cea073 None admin] PATH_INFO: `/v3/projects/15d53acb4e8c43348673ba66109e0371` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:45.492352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe7e5e-eef1-47a6-940c-521473cea073 None admin] RBAC: Authorizing `identity:get_project(project_id=15d53acb4e8c43348673ba66109e0371)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:45.494432 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe7e5e-eef1-47a6-940c-521473cea073 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:45.496592 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 135/262] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:45 2022] GET /identity/v3/projects/15d53acb4e8c43348673ba66109e0371 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:41:45.503106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe7e5e-eef1-47a6-940c-521473cea073 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:45.547475 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a0471-2fad-44c0-8166-bc8e5bc104f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:45.548039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a0471-2fad-44c0-8166-bc8e5bc104f9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:45.548325 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a0471-2fad-44c0-8166-bc8e5bc104f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:45.548598 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a0471-2fad-44c0-8166-bc8e5bc104f9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:45.578178 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3a0471-2fad-44c0-8166-bc8e5bc104f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:45.584143 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-fd3a0471-2fad-44c0-8166-bc8e5bc104f9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 23:41:45.584143 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:45.584143 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:45.584143 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:45.584143 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:45.584143 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:45.584143 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:45.584143 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:45.584143 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:45.584143 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:45.584143 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:45.584143 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:45.584143 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:45.584143 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:45.584143 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:45.584143 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:45.585008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:45.585008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:45.585008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:45.585008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:45.585008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:45.585008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:45.585008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:45.585008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:45.585008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:45.585008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:45.585008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:45.585008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:45.585008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:45.585008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:45.585008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:45.585008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:45.585766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:45.585766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:45.585766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:45.585766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:45.585766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:45.585766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:45.585766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:45.585766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:45.585766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:45.585766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:45.585766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 23:41:45.585766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:41:45.585766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 128/263] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Sep 10 23:41:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:41:45.595962 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a0471-2fad-44c0-8166-bc8e5bc104f9 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:45.652502 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe7e5e-eef1-47a6-940c-521473cea073 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:45.688650 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f240d4e-bbf3-42a8-aec4-5155fc28735b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:45.689607 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f240d4e-bbf3-42a8-aec4-5155fc28735b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:45.690072 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f240d4e-bbf3-42a8-aec4-5155fc28735b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:45.690550 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f240d4e-bbf3-42a8-aec4-5155fc28735b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:45.691481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f240d4e-bbf3-42a8-aec4-5155fc28735b None admin] RBAC: Adding query filter params (role.id=9dce48a2b53c482d931dcd064b07744e, scope.project.id=15d53acb4e8c43348673ba66109e0371, user.id=61fe9e83ae764584964d9139676ed85e) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:45.691991 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f240d4e-bbf3-42a8-aec4-5155fc28735b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:45.693621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f240d4e-bbf3-42a8-aec4-5155fc28735b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:45.701000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 136/264] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:41:45 2022] GET /identity/v3/role_assignments?user.id=61fe9e83ae764584964d9139676ed85e&scope.project.id=15d53acb4e8c43348673ba66109e0371&role.id=9dce48a2b53c482d931dcd064b07744e => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:45.741299 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:45.813580 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a0471-2fad-44c0-8166-bc8e5bc104f9 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:45.860360 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dfafb7-4ebf-4995-aafa-250f20cbd48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:45.860983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dfafb7-4ebf-4995-aafa-250f20cbd48a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:45.861257 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dfafb7-4ebf-4995-aafa-250f20cbd48a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:45.861522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dfafb7-4ebf-4995-aafa-250f20cbd48a None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:45.862196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dfafb7-4ebf-4995-aafa-250f20cbd48a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:45.862496 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dfafb7-4ebf-4995-aafa-250f20cbd48a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:45.863901 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dfafb7-4ebf-4995-aafa-250f20cbd48a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:45.879233 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 129/265] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:41:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:41:45.885257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f240d4e-bbf3-42a8-aec4-5155fc28735b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:45.958235 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:46.037379 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f240d4e-bbf3-42a8-aec4-5155fc28735b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:46.079978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec2753-53cb-4c80-87ed-2f56feca1027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:46.080539 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec2753-53cb-4c80-87ed-2f56feca1027 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:46.080819 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec2753-53cb-4c80-87ed-2f56feca1027 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:46.081095 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec2753-53cb-4c80-87ed-2f56feca1027 None admin] PATH_INFO: `/v3/users/61fe9e83ae764584964d9139676ed85e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:46.082262 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec2753-53cb-4c80-87ed-2f56feca1027 None admin] RBAC: Authorizing `identity:get_user(user_id=61fe9e83ae764584964d9139676ed85e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:46.083884 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec2753-53cb-4c80-87ed-2f56feca1027 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:46.086759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 137/266] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:45 2022] GET /identity/v3/users/61fe9e83ae764584964d9139676ed85e => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:46.091987 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dfafb7-4ebf-4995-aafa-250f20cbd48a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:46.243256 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dfafb7-4ebf-4995-aafa-250f20cbd48a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:46.265298 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:46.285820 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc88eb-f68c-4011-a8f0-b4a87f60013b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:46.286433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc88eb-f68c-4011-a8f0-b4a87f60013b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:46.286732 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc88eb-f68c-4011-a8f0-b4a87f60013b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:46.287004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc88eb-f68c-4011-a8f0-b4a87f60013b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:46.384822 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dc88eb-f68c-4011-a8f0-b4a87f60013b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:46.386215 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dc88eb-f68c-4011-a8f0-b4a87f60013b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:46.507580 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 130/267] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:41:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:41:46.513102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec2753-53cb-4c80-87ed-2f56feca1027 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:46.692645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec2753-53cb-4c80-87ed-2f56feca1027 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:46.706465 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:46.739694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebdc04-b385-4dfa-a353-d8228e1fbfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:46.740381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebdc04-b385-4dfa-a353-d8228e1fbfa7 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:46.740660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebdc04-b385-4dfa-a353-d8228e1fbfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:46.740932 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebdc04-b385-4dfa-a353-d8228e1fbfa7 None admin] PATH_INFO: `/v3/projects/15d53acb4e8c43348673ba66109e0371` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:46.742056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ebdc04-b385-4dfa-a353-d8228e1fbfa7 None admin] RBAC: Authorizing `identity:get_project(project_id=15d53acb4e8c43348673ba66109e0371)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:46.743559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ebdc04-b385-4dfa-a353-d8228e1fbfa7 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:46.748925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 138/268] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:46 2022] GET /identity/v3/projects/15d53acb4e8c43348673ba66109e0371 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:46.750608 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a719e7-7d68-4186-be75-237b7ca4f76b None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:46.750935 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a719e7-7d68-4186-be75-237b7ca4f76b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:46.751207 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a719e7-7d68-4186-be75-237b7ca4f76b None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:46.753014 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebdc04-b385-4dfa-a353-d8228e1fbfa7 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:46.753267 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 131/269] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:46.759022 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575668d-0627-4896-baa1-49f906b9e183 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:46.759302 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575668d-0627-4896-baa1-49f906b9e183 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:46.759568 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575668d-0627-4896-baa1-49f906b9e183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:46.806130 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-9575668d-0627-4896-baa1-49f906b9e183 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:46.807642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9575668d-0627-4896-baa1-49f906b9e183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:46.849212 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:46.880249 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebdc04-b385-4dfa-a353-d8228e1fbfa7 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:46.918606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 132/270] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 23:41:46.922708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b6447-76dc-48fa-b011-3d396ec44c1b None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:46.922708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b6447-76dc-48fa-b011-3d396ec44c1b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:46.922708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b6447-76dc-48fa-b011-3d396ec44c1b None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:46.922708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 133/271] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:46.928119 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1af75f-7c28-4e3a-9eb2-ef17bad0b070 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:46.928404 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1af75f-7c28-4e3a-9eb2-ef17bad0b070 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:46.928687 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1af75f-7c28-4e3a-9eb2-ef17bad0b070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:46.933808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce7a8b-8be8-4fa1-aa02-47e3552d7338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:46.934419 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce7a8b-8be8-4fa1-aa02-47e3552d7338 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:46.934784 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce7a8b-8be8-4fa1-aa02-47e3552d7338 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:46.935066 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce7a8b-8be8-4fa1-aa02-47e3552d7338 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:46.935800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce7a8b-8be8-4fa1-aa02-47e3552d7338 None admin] RBAC: Adding query filter params (role.id=ff2ed192587843e5b10b5a35949e4c5f, scope.project.id=15d53acb4e8c43348673ba66109e0371, user.id=61fe9e83ae764584964d9139676ed85e) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:46.936110 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce7a8b-8be8-4fa1-aa02-47e3552d7338 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:46.937567 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce7a8b-8be8-4fa1-aa02-47e3552d7338 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:46.944045 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 139/272] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:41:46 2022] GET /identity/v3/role_assignments?user.id=61fe9e83ae764584964d9139676ed85e&scope.project.id=15d53acb4e8c43348673ba66109e0371&role.id=ff2ed192587843e5b10b5a35949e4c5f => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:41:46.957808 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-3b1af75f-7c28-4e3a-9eb2-ef17bad0b070 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:46.959087 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3b1af75f-7c28-4e3a-9eb2-ef17bad0b070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:46.967012 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acce1c-9b9a-4955-9b48-04bc1b216d55 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:46.967321 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acce1c-9b9a-4955-9b48-04bc1b216d55 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:46.967574 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acce1c-9b9a-4955-9b48-04bc1b216d55 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:46.969157 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 140/273] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:46.979787 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb2602-85ca-49a8-bd75-d8d314fa4dbe None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:46.980170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb2602-85ca-49a8-bd75-d8d314fa4dbe None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:46.980452 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb2602-85ca-49a8-bd75-d8d314fa4dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:47.021455 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-4fbb2602-85ca-49a8-bd75-d8d314fa4dbe None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:47.022763 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4fbb2602-85ca-49a8-bd75-d8d314fa4dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:47.077523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:47.103149 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 134/274] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 23:41:47.130848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 141/275] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:47.210175 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1af75f-7c28-4e3a-9eb2-ef17bad0b070 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:47.321464 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:47.341974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1af75f-7c28-4e3a-9eb2-ef17bad0b070 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:47.379917 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94775fa-9e32-489a-b10d-23af50fa1585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:47.380603 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94775fa-9e32-489a-b10d-23af50fa1585 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:47.380879 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94775fa-9e32-489a-b10d-23af50fa1585 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:47.381151 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94775fa-9e32-489a-b10d-23af50fa1585 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:47.392053 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94775fa-9e32-489a-b10d-23af50fa1585 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:47.396956 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-d94775fa-9e32-489a-b10d-23af50fa1585 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 23:41:47.396956 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:47.396956 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:47.396956 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:47.396956 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:47.396956 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:47.396956 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:47.396956 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:47.396956 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:47.396956 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:47.396956 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:47.396956 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:47.396956 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:47.396956 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:47.396956 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:47.396956 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:47.397812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:47.397812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:47.397812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:47.397812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:47.397812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:47.397812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:47.397812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:47.397812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:47.397812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:47.397812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:47.397812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:47.397812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:47.397812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:47.397812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:47.397812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:47.397812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:47.399002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:47.399002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:47.399002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:47.399002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:47.399002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:47.399002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:47.399002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:47.399002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:47.399002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:47.399002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:47.399002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 23:41:47.399002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:41:47.399002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 135/276] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:47.404924 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb2602-85ca-49a8-bd75-d8d314fa4dbe None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:47.424575 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94775fa-9e32-489a-b10d-23af50fa1585 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:47.536277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb2602-85ca-49a8-bd75-d8d314fa4dbe None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:47.552512 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94775fa-9e32-489a-b10d-23af50fa1585 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:47.575710 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30416969-f59d-4fee-a770-4ec135b854d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:47.576543 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30416969-f59d-4fee-a770-4ec135b854d0 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:47.576655 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30416969-f59d-4fee-a770-4ec135b854d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:47.576844 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30416969-f59d-4fee-a770-4ec135b854d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:47.577501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30416969-f59d-4fee-a770-4ec135b854d0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:47.577806 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30416969-f59d-4fee-a770-4ec135b854d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:47.579397 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30416969-f59d-4fee-a770-4ec135b854d0 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:47.589008 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 142/277] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:41:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:47.589655 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30416969-f59d-4fee-a770-4ec135b854d0 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:47.595354 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559339a7-8292-4e08-8675-638c8b6aaf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:47.595986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559339a7-8292-4e08-8675-638c8b6aaf7d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:47.596259 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559339a7-8292-4e08-8675-638c8b6aaf7d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:47.596525 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559339a7-8292-4e08-8675-638c8b6aaf7d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:47.597627 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559339a7-8292-4e08-8675-638c8b6aaf7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:47.599103 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559339a7-8292-4e08-8675-638c8b6aaf7d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:47.600667 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 136/278] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:41:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:41:47.601414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559339a7-8292-4e08-8675-638c8b6aaf7d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:47.614385 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:47.631574 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:47.699894 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30416969-f59d-4fee-a770-4ec135b854d0 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:47.706523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559339a7-8292-4e08-8675-638c8b6aaf7d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:47.736237 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506c787-2faa-4173-8081-0efc8c93aa70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:47.736871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506c787-2faa-4173-8081-0efc8c93aa70 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:47.737146 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506c787-2faa-4173-8081-0efc8c93aa70 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:47.737428 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506c787-2faa-4173-8081-0efc8c93aa70 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:47.740485 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d16dc5-d3d9-4dbd-99e8-c1fc3e1ea4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:47.741104 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d16dc5-d3d9-4dbd-99e8-c1fc3e1ea4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:47.741381 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d16dc5-d3d9-4dbd-99e8-c1fc3e1ea4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:47.741650 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d16dc5-d3d9-4dbd-99e8-c1fc3e1ea4b7 None admin] PATH_INFO: `/v3/users/67b2fb0ba6574c42a17448654d297564` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:47.742798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d16dc5-d3d9-4dbd-99e8-c1fc3e1ea4b7 None admin] RBAC: Authorizing `identity:get_user(user_id=67b2fb0ba6574c42a17448654d297564)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:47.744264 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d16dc5-d3d9-4dbd-99e8-c1fc3e1ea4b7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:47.745835 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 137/279] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:47 2022] GET /identity/v3/users/67b2fb0ba6574c42a17448654d297564 => generated 297 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:47.750878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d16dc5-d3d9-4dbd-99e8-c1fc3e1ea4b7 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:47.751939 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506c787-2faa-4173-8081-0efc8c93aa70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:47.757295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-6506c787-2faa-4173-8081-0efc8c93aa70 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:47.757295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:47.757295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:47.757295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:47.757295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:47.757295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:47.757295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:47.757295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:47.757295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:47.757295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:47.757295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:47.757295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:47.757295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:47.757295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:47.757295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:47.757295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:47.758176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:47.758176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:47.758176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:47.758176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:47.758176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:47.758176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:47.758176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:47.758176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:47.758176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:47.758176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:47.758176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:47.758176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:47.758176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:47.758176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:47.758176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:47.758176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:47.758999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:47.758999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:47.758999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:47.758999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:47.758999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:47.758999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:47.758999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:47.758999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:47.758999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:47.758999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:47.758999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:47.758999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:41:47.758999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 143/280] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:47.759638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506c787-2faa-4173-8081-0efc8c93aa70 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:47.842131 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:47.864319 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d16dc5-d3d9-4dbd-99e8-c1fc3e1ea4b7 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:47.874995 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:47.883545 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506c787-2faa-4173-8081-0efc8c93aa70 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:47.899780 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295de551-d34f-4f8e-9b90-2ffb10d6dea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:47.900378 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295de551-d34f-4f8e-9b90-2ffb10d6dea6 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:47.900687 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295de551-d34f-4f8e-9b90-2ffb10d6dea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:47.901081 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295de551-d34f-4f8e-9b90-2ffb10d6dea6 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:47.901766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295de551-d34f-4f8e-9b90-2ffb10d6dea6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:47.903245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295de551-d34f-4f8e-9b90-2ffb10d6dea6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:47.921055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c0d27c-b92f-40b9-bb27-0cb5c8b7940a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:47.921681 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c0d27c-b92f-40b9-bb27-0cb5c8b7940a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:47.921950 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c0d27c-b92f-40b9-bb27-0cb5c8b7940a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:47.923874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c0d27c-b92f-40b9-bb27-0cb5c8b7940a None admin] PATH_INFO: `/v3/projects/15d53acb4e8c43348673ba66109e0371` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:47.923874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c0d27c-b92f-40b9-bb27-0cb5c8b7940a None admin] RBAC: Authorizing `identity:get_project(project_id=15d53acb4e8c43348673ba66109e0371)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:47.924977 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c0d27c-b92f-40b9-bb27-0cb5c8b7940a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:47.926591 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 144/281] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:47 2022] GET /identity/v3/projects/15d53acb4e8c43348673ba66109e0371 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:47.927410 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c0d27c-b92f-40b9-bb27-0cb5c8b7940a None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:48.000384 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 138/282] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Sep 10 23:41:47 2022] POST /identity/v3/users => generated 293 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:41:48.001681 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5a73f-be15-4ac6-a67e-4f8490d701b3 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:48.001961 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5a73f-be15-4ac6-a67e-4f8490d701b3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:48.002238 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5a73f-be15-4ac6-a67e-4f8490d701b3 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:48.003494 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 139/283] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:48.008136 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5a73f-be15-4ac6-a67e-4f8490d701b3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:48.036690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c0d27c-b92f-40b9-bb27-0cb5c8b7940a None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:48.076642 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc4397-f9d9-4c01-b5bc-ae101468f00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:48.077345 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc4397-f9d9-4c01-b5bc-ae101468f00b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:48.077627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc4397-f9d9-4c01-b5bc-ae101468f00b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:48.077899 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc4397-f9d9-4c01-b5bc-ae101468f00b None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:48.082214 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc4397-f9d9-4c01-b5bc-ae101468f00b None admin] RBAC: Adding query filter params (name=member) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:48.082516 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc4397-f9d9-4c01-b5bc-ae101468f00b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:48.085104 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc4397-f9d9-4c01-b5bc-ae101468f00b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:48.099003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 145/284] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:41:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:48.099003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38034fec-45a3-4d3d-b422-95567bdd87c0 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:48.099003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38034fec-45a3-4d3d-b422-95567bdd87c0 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:48.099003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38034fec-45a3-4d3d-b422-95567bdd87c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:48.115990 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-38034fec-45a3-4d3d-b422-95567bdd87c0 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:48.117422 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-38034fec-45a3-4d3d-b422-95567bdd87c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:48.140820 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5a73f-be15-4ac6-a67e-4f8490d701b3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:48.141344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:48.176866 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:48.184007 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b06a2b-513b-4d0e-8d6c-7b9590ff536b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:48.185234 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b06a2b-513b-4d0e-8d6c-7b9590ff536b None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:48.185508 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b06a2b-513b-4d0e-8d6c-7b9590ff536b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:48.185937 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b06a2b-513b-4d0e-8d6c-7b9590ff536b None admin] PATH_INFO: `/v3/projects/15d53acb4e8c43348673ba66109e0371/users/67b2fb0ba6574c42a17448654d297564/roles/900ce80935da4fa59be9334c5c8e0083` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:48.187871 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b06a2b-513b-4d0e-8d6c-7b9590ff536b None admin] RBAC: Authorizing `identity:create_grant(project_id=15d53acb4e8c43348673ba66109e0371, user_id=67b2fb0ba6574c42a17448654d297564, role_id=900ce80935da4fa59be9334c5c8e0083)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:48.189371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b06a2b-513b-4d0e-8d6c-7b9590ff536b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:48.197944 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 146/285] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:48.198941 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38034fec-45a3-4d3d-b422-95567bdd87c0 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:48.262368 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 140/286] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Sep 10 23:41:48 2022] PUT /identity/v3/projects/15d53acb4e8c43348673ba66109e0371/users/67b2fb0ba6574c42a17448654d297564/roles/900ce80935da4fa59be9334c5c8e0083 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:41:48.338406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38034fec-45a3-4d3d-b422-95567bdd87c0 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:48.380194 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1fd5c-95d3-42a6-8636-b00f0b15d7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:48.380743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1fd5c-95d3-42a6-8636-b00f0b15d7a0 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:48.381069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1fd5c-95d3-42a6-8636-b00f0b15d7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:48.381330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1fd5c-95d3-42a6-8636-b00f0b15d7a0 None admin] PATH_INFO: `/v3/users/df486f35f7a540c385f0bc07ccd7cc26` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:48.398920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1fd5c-95d3-42a6-8636-b00f0b15d7a0 None admin] RBAC: Authorizing `identity:get_user(user_id=df486f35f7a540c385f0bc07ccd7cc26)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:48.400389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1fd5c-95d3-42a6-8636-b00f0b15d7a0 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:48.402480 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 147/287] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:48 2022] GET /identity/v3/users/df486f35f7a540c385f0bc07ccd7cc26 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:48.408295 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b06a2b-513b-4d0e-8d6c-7b9590ff536b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:48.501043 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1fd5c-95d3-42a6-8636-b00f0b15d7a0 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:48.522496 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:48.525471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b06a2b-513b-4d0e-8d6c-7b9590ff536b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:48.571818 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f938726-d42a-4fc7-81e4-ccbfd5d935e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:48.572424 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f938726-d42a-4fc7-81e4-ccbfd5d935e4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:48.572695 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f938726-d42a-4fc7-81e4-ccbfd5d935e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:48.572960 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f938726-d42a-4fc7-81e4-ccbfd5d935e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:48.573832 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f938726-d42a-4fc7-81e4-ccbfd5d935e4 None admin] RBAC: Adding query filter params (role.id=824f5fe1f02b447382190a8c236c27db, user.id=df486f35f7a540c385f0bc07ccd7cc26, scope.system=all) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:48.574138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f938726-d42a-4fc7-81e4-ccbfd5d935e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:48.576265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f938726-d42a-4fc7-81e4-ccbfd5d935e4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:48.582216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 141/288] 10.222.0.17 () {56 vars in 1308 bytes} [Sat Sep 10 23:41:48 2022] GET /identity/v3/role_assignments?user.id=df486f35f7a540c385f0bc07ccd7cc26&scope.system=all&role.id=824f5fe1f02b447382190a8c236c27db => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:48.626098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1fd5c-95d3-42a6-8636-b00f0b15d7a0 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:48.679156 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8045e0d-2ad1-4961-84e1-db9de6d4f258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:48.679808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8045e0d-2ad1-4961-84e1-db9de6d4f258 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:48.680084 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8045e0d-2ad1-4961-84e1-db9de6d4f258 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:48.680356 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8045e0d-2ad1-4961-84e1-db9de6d4f258 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:48.695678 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8045e0d-2ad1-4961-84e1-db9de6d4f258 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:48.701437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-d8045e0d-2ad1-4961-84e1-db9de6d4f258 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:48.701437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:48.701437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:48.701437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:48.701437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:48.701437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:48.701437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:48.701437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:48.701437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:48.701437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:48.701437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:48.701437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:48.701437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:48.701437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:48.701437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:48.701437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:48.702371 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:48.702371 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:48.702371 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:48.702371 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:48.702371 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:48.702371 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:48.702371 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:48.702371 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:48.702371 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:48.702371 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:48.702371 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:48.702371 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:48.702371 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:48.702371 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:48.702371 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:48.702371 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:48.703778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:48.703778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:48.703778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:48.703778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:48.703778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:48.703778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:48.703778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:48.703778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:48.703778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:48.703778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:48.703778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:48.703778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:41:48.703778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 148/289] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:48.708593 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f938726-d42a-4fc7-81e4-ccbfd5d935e4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:48.790978 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:48.857203 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f938726-d42a-4fc7-81e4-ccbfd5d935e4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:48.899071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4183e-cab8-4942-bcf2-c1bd964105f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:48.899766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4183e-cab8-4942-bcf2-c1bd964105f4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:48.899986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4183e-cab8-4942-bcf2-c1bd964105f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:48.900201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4183e-cab8-4942-bcf2-c1bd964105f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:48.900865 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4183e-cab8-4942-bcf2-c1bd964105f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:48.901160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4183e-cab8-4942-bcf2-c1bd964105f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:48.902635 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4183e-cab8-4942-bcf2-c1bd964105f4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:48.912442 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 142/290] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:41:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:48.917827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8045e0d-2ad1-4961-84e1-db9de6d4f258 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:49.075357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:49.106783 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8045e0d-2ad1-4961-84e1-db9de6d4f258 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:49.156913 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f80041-5c9e-418e-a73b-7b24f0b00c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:49.157452 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f80041-5c9e-418e-a73b-7b24f0b00c37 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:49.157768 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f80041-5c9e-418e-a73b-7b24f0b00c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:49.158010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f80041-5c9e-418e-a73b-7b24f0b00c37 None admin] PATH_INFO: `/v3/users/d410102c90b843c691b4401ffb232f68` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:49.159239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f80041-5c9e-418e-a73b-7b24f0b00c37 None admin] RBAC: Authorizing `identity:get_user(user_id=d410102c90b843c691b4401ffb232f68)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:49.160705 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f80041-5c9e-418e-a73b-7b24f0b00c37 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:49.162617 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 149/291] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:48 2022] GET /identity/v3/users/d410102c90b843c691b4401ffb232f68 => generated 297 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:49.167903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4183e-cab8-4942-bcf2-c1bd964105f4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:49.323379 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4183e-cab8-4942-bcf2-c1bd964105f4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:49.342711 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:49.360382 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e50ce-dbe4-4d5d-908c-05ccef875604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:49.360996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e50ce-dbe4-4d5d-908c-05ccef875604 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:49.361281 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e50ce-dbe4-4d5d-908c-05ccef875604 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:49.361553 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e50ce-dbe4-4d5d-908c-05ccef875604 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:49.463981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e50ce-dbe4-4d5d-908c-05ccef875604 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:49.465450 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e50ce-dbe4-4d5d-908c-05ccef875604 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:49.602885 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 143/292] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:41:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:41:49.608781 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f80041-5c9e-418e-a73b-7b24f0b00c37 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:49.762928 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f80041-5c9e-418e-a73b-7b24f0b00c37 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:49.815151 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efffd7f-9615-41d1-887d-af7420d4fb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:49.816162 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efffd7f-9615-41d1-887d-af7420d4fb2d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:49.816636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efffd7f-9615-41d1-887d-af7420d4fb2d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:49.817118 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efffd7f-9615-41d1-887d-af7420d4fb2d None admin] PATH_INFO: `/v3/projects/15d53acb4e8c43348673ba66109e0371` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:49.818420 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efffd7f-9615-41d1-887d-af7420d4fb2d None admin] RBAC: Authorizing `identity:get_project(project_id=15d53acb4e8c43348673ba66109e0371)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:49.820079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efffd7f-9615-41d1-887d-af7420d4fb2d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:49.821897 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 150/293] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:49 2022] GET /identity/v3/projects/15d53acb4e8c43348673ba66109e0371 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:49.828083 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e50ce-dbe4-4d5d-908c-05ccef875604 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:49.859458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:49.964209 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e50ce-dbe4-4d5d-908c-05ccef875604 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:50.021217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2929e1d-40b9-49c2-a53a-48e1a88615ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:50.021868 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2929e1d-40b9-49c2-a53a-48e1a88615ac None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:50.022146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2929e1d-40b9-49c2-a53a-48e1a88615ac None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:50.022411 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2929e1d-40b9-49c2-a53a-48e1a88615ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:50.023160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2929e1d-40b9-49c2-a53a-48e1a88615ac None admin] RBAC: Adding query filter params (role.id=824f5fe1f02b447382190a8c236c27db, scope.project.id=15d53acb4e8c43348673ba66109e0371, user.id=d410102c90b843c691b4401ffb232f68) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:50.023460 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2929e1d-40b9-49c2-a53a-48e1a88615ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:50.024974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2929e1d-40b9-49c2-a53a-48e1a88615ac None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:50.033117 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 144/294] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:41:49 2022] GET /identity/v3/role_assignments?user.id=d410102c90b843c691b4401ffb232f68&scope.project.id=15d53acb4e8c43348673ba66109e0371&role.id=824f5fe1f02b447382190a8c236c27db => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:50.822330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85292a2-ca19-4cf9-ba52-def82ddacfa1 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:50.822904 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85292a2-ca19-4cf9-ba52-def82ddacfa1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:50.822904 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85292a2-ca19-4cf9-ba52-def82ddacfa1 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:50.824621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 151/295] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:50.830726 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd67b3-fea8-49cf-b214-85283332c339 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:50.830981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd67b3-fea8-49cf-b214-85283332c339 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:50.831244 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd67b3-fea8-49cf-b214-85283332c339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:50.852181 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-facd67b3-fea8-49cf-b214-85283332c339 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:50.853709 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-facd67b3-fea8-49cf-b214-85283332c339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:50.898605 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cacd9-ee2e-44dd-b0a0-ae7fb2282f18 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:50.898942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cacd9-ee2e-44dd-b0a0-ae7fb2282f18 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:50.899189 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cacd9-ee2e-44dd-b0a0-ae7fb2282f18 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:50.900584 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 152/296] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:50.907090 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3901e82-7cdd-46a2-88fe-8da3739210f1 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:50.907525 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3901e82-7cdd-46a2-88fe-8da3739210f1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:50.907905 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3901e82-7cdd-46a2-88fe-8da3739210f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:50.929805 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-f3901e82-7cdd-46a2-88fe-8da3739210f1 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:50.931196 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f3901e82-7cdd-46a2-88fe-8da3739210f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:50.933922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 145/297] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:50.934753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:51.004017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 153/298] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:51.021574 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0afaf-f0ce-494f-9cef-0d4e2128fcc0 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:51.022522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0afaf-f0ce-494f-9cef-0d4e2128fcc0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:51.023233 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0afaf-f0ce-494f-9cef-0d4e2128fcc0 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:51.024860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 146/299] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:51.031420 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7360ee9-66cf-45bf-bf58-12e673991a15 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:51.031939 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7360ee9-66cf-45bf-bf58-12e673991a15 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:51.032408 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7360ee9-66cf-45bf-bf58-12e673991a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:51.051491 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b7360ee9-66cf-45bf-bf58-12e673991a15 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:51.052994 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b7360ee9-66cf-45bf-bf58-12e673991a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:51.119853 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 154/300] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:51.219757 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0afaf-f0ce-494f-9cef-0d4e2128fcc0 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:51.235238 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:51.284376 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7360ee9-66cf-45bf-bf58-12e673991a15 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:51.348201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0afaf-f0ce-494f-9cef-0d4e2128fcc0 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:51.377167 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:51.384966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b71d2d-b348-4194-bcc2-eb4d92bb07b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:51.385589 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b71d2d-b348-4194-bcc2-eb4d92bb07b4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:51.385873 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b71d2d-b348-4194-bcc2-eb4d92bb07b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:51.390959 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b71d2d-b348-4194-bcc2-eb4d92bb07b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:51.390959 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b71d2d-b348-4194-bcc2-eb4d92bb07b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:51.390959 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b71d2d-b348-4194-bcc2-eb4d92bb07b4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:51.391422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 147/301] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:41:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:51.392423 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da08a0-3ad2-4da7-b4b5-0903d86cd4eb None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:51.392702 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da08a0-3ad2-4da7-b4b5-0903d86cd4eb None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:51.393055 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da08a0-3ad2-4da7-b4b5-0903d86cd4eb None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:51.394292 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 148/302] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:51.399036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da08a0-3ad2-4da7-b4b5-0903d86cd4eb None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:51.409588 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7360ee9-66cf-45bf-bf58-12e673991a15 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:51.444408 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e8ec2-39b3-4480-93b2-c201ae848e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:51.445033 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e8ec2-39b3-4480-93b2-c201ae848e97 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:51.445341 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e8ec2-39b3-4480-93b2-c201ae848e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:51.445673 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e8ec2-39b3-4480-93b2-c201ae848e97 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:51.456067 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648e8ec2-39b3-4480-93b2-c201ae848e97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:51.460880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-648e8ec2-39b3-4480-93b2-c201ae848e97 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 23:41:51.460880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:51.460880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:51.460880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:51.460880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:51.460880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:51.460880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:51.460880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:51.460880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:51.460880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:51.460880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:51.460880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:51.460880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:51.460880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:51.460880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:51.460880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:51.461730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:51.461730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:51.461730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:51.461730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:51.461730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:51.461730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:51.461730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:51.461730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:51.461730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:51.461730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:51.461730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:51.461730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:51.461730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:51.461730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:51.461730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:51.461730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:51.462514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:51.462514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:51.462514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:51.462514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:51.462514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:51.462514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:51.462514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:51.462514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:51.462514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:51.462514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:51.462514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 23:41:51.462514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:41:51.462514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 155/303] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:51.463861 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f60c08-62e5-43d1-9bc0-e7d174accf50 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:51.464166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f60c08-62e5-43d1-9bc0-e7d174accf50 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:51.464439 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f60c08-62e5-43d1-9bc0-e7d174accf50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:51.482325 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-02f60c08-62e5-43d1-9bc0-e7d174accf50 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:51.482876 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:51.483555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-02f60c08-62e5-43d1-9bc0-e7d174accf50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:51.504180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da08a0-3ad2-4da7-b4b5-0903d86cd4eb None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:51.537530 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40676ca6-5a29-4268-b8cf-1db3bb213b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:51.538129 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40676ca6-5a29-4268-b8cf-1db3bb213b0c None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:51.538407 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40676ca6-5a29-4268-b8cf-1db3bb213b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:51.538702 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40676ca6-5a29-4268-b8cf-1db3bb213b0c None admin] PATH_INFO: `/v3/groups` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:51.539332 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40676ca6-5a29-4268-b8cf-1db3bb213b0c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:51.540767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40676ca6-5a29-4268-b8cf-1db3bb213b0c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:51.549395 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 156/304] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:51.550233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f60c08-62e5-43d1-9bc0-e7d174accf50 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:51.607632 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:51.611611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 149/305] 10.222.0.17 () {60 vars in 1134 bytes} [Sat Sep 10 23:41:51 2022] POST /identity/v3/groups => generated 229 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:41:51.612443 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40676ca6-5a29-4268-b8cf-1db3bb213b0c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:51.673555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f60c08-62e5-43d1-9bc0-e7d174accf50 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:51.715924 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c1dc4-2c3d-4902-96b4-398ad5f8a85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:51.716981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c1dc4-2c3d-4902-96b4-398ad5f8a85c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:51.717454 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c1dc4-2c3d-4902-96b4-398ad5f8a85c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:51.717935 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c1dc4-2c3d-4902-96b4-398ad5f8a85c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:51.731949 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c1dc4-2c3d-4902-96b4-398ad5f8a85c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:51.739356 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-088c1dc4-2c3d-4902-96b4-398ad5f8a85c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 23:41:51.739356 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:51.739356 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:51.739356 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:51.739356 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:51.739356 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:51.739356 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:51.739356 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:51.739356 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:51.739356 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:51.739356 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:51.739356 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:51.739356 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:51.739356 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:51.739356 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:51.739356 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:51.740292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:51.740292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:51.740292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:51.740292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:51.740292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:51.740292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:51.740292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:51.740292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:51.740292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:51.740292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:51.740292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:51.740292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:51.740292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:51.740292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:51.740292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:51.740292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:51.741221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:51.741221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:51.741221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:51.741221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:51.741221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:51.741221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:51.741221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:51.741221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:51.741221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:51.741221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:51.741221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 23:41:51.741221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:41:51.744738 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 157/306] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:51.757425 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40676ca6-5a29-4268-b8cf-1db3bb213b0c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:51.757631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c1dc4-2c3d-4902-96b4-398ad5f8a85c None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:51.796461 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6f369-b594-4103-8b39-d129eeb6b102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:51.796848 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6f369-b594-4103-8b39-d129eeb6b102 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:51.797128 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6f369-b594-4103-8b39-d129eeb6b102 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:51.797405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6f369-b594-4103-8b39-d129eeb6b102 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:51.798107 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6f369-b594-4103-8b39-d129eeb6b102 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:51.798408 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6f369-b594-4103-8b39-d129eeb6b102 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:51.799979 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6f369-b594-4103-8b39-d129eeb6b102 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:51.809225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 150/307] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:41:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:51.816350 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6f369-b594-4103-8b39-d129eeb6b102 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:51.820473 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:51.881780 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c1dc4-2c3d-4902-96b4-398ad5f8a85c None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:51.902444 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:51.917576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568f982-3095-4d3f-abdc-b2aaf1bf9716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:51.918270 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568f982-3095-4d3f-abdc-b2aaf1bf9716 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:51.918551 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568f982-3095-4d3f-abdc-b2aaf1bf9716 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:51.918886 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568f982-3095-4d3f-abdc-b2aaf1bf9716 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:51.920107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568f982-3095-4d3f-abdc-b2aaf1bf9716 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:51.920411 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568f982-3095-4d3f-abdc-b2aaf1bf9716 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:51.921818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568f982-3095-4d3f-abdc-b2aaf1bf9716 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:51.926310 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6f369-b594-4103-8b39-d129eeb6b102 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:51.935396 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 158/308] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:41:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:51.935396 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568f982-3095-4d3f-abdc-b2aaf1bf9716 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:51.967049 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95134f-a1ad-4651-a163-70cfcf1ddda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:51.967673 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95134f-a1ad-4651-a163-70cfcf1ddda8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:51.967949 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95134f-a1ad-4651-a163-70cfcf1ddda8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:51.968218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95134f-a1ad-4651-a163-70cfcf1ddda8 None admin] PATH_INFO: `/v3/users/31a463346db64e0dabd65403016fbe67` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:51.983314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f95134f-a1ad-4651-a163-70cfcf1ddda8 None admin] RBAC: Authorizing `identity:get_user(user_id=31a463346db64e0dabd65403016fbe67)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:51.984906 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f95134f-a1ad-4651-a163-70cfcf1ddda8 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:51.986780 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 151/309] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:51 2022] GET /identity/v3/users/31a463346db64e0dabd65403016fbe67 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:41:51.987425 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95134f-a1ad-4651-a163-70cfcf1ddda8 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:52.049660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568f982-3095-4d3f-abdc-b2aaf1bf9716 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:52.058609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:52.090457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578bde5-1c1b-4f7a-aee0-916623a19561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:52.091276 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578bde5-1c1b-4f7a-aee0-916623a19561 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:52.091548 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578bde5-1c1b-4f7a-aee0-916623a19561 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:52.091812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578bde5-1c1b-4f7a-aee0-916623a19561 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:52.103513 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95134f-a1ad-4651-a163-70cfcf1ddda8 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:52.103647 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578bde5-1c1b-4f7a-aee0-916623a19561 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:52.108997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-d578bde5-1c1b-4f7a-aee0-916623a19561 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:52.108997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:52.108997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:52.108997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:52.108997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:52.108997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:52.108997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:52.108997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:52.108997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:52.108997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:52.108997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:52.108997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:52.108997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:52.108997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:52.108997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:52.108997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:52.109863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:52.109863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:52.109863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:52.109863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:52.109863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:52.109863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:52.109863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:52.109863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:52.109863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:52.109863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:52.109863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:52.109863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:52.109863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:52.109863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:52.109863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:52.109863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:52.110635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:52.110635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:52.110635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:52.110635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:52.110635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:52.110635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:52.110635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:52.110635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:52.110635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:52.110635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:52.110635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:52.110635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:41:52.110635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 159/310] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:52.111818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578bde5-1c1b-4f7a-aee0-916623a19561 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:52.141373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325aabf-6741-4576-a2bb-387d8ecabdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:52.141990 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325aabf-6741-4576-a2bb-387d8ecabdc5 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:52.142262 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325aabf-6741-4576-a2bb-387d8ecabdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:52.142526 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325aabf-6741-4576-a2bb-387d8ecabdc5 None admin] PATH_INFO: `/v3/users/67b2fb0ba6574c42a17448654d297564` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:52.143692 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b325aabf-6741-4576-a2bb-387d8ecabdc5 None admin] RBAC: Authorizing `identity:get_user(user_id=67b2fb0ba6574c42a17448654d297564)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:52.145168 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b325aabf-6741-4576-a2bb-387d8ecabdc5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:52.146901 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 152/311] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:51 2022] GET /identity/v3/users/67b2fb0ba6574c42a17448654d297564 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:52.151790 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325aabf-6741-4576-a2bb-387d8ecabdc5 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:52.165571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:52.229574 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578bde5-1c1b-4f7a-aee0-916623a19561 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:52.260855 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325aabf-6741-4576-a2bb-387d8ecabdc5 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:52.268116 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0dc441-bf5c-4207-8b34-0d213130391d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:52.268963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dc441-bf5c-4207-8b34-0d213130391d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:52.269429 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dc441-bf5c-4207-8b34-0d213130391d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:52.269899 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dc441-bf5c-4207-8b34-0d213130391d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:52.270834 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0dc441-bf5c-4207-8b34-0d213130391d None admin] RBAC: Adding query filter params (role.id=900ce80935da4fa59be9334c5c8e0083, user.id=31a463346db64e0dabd65403016fbe67, scope.system=all) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:52.271344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0dc441-bf5c-4207-8b34-0d213130391d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:52.272952 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0dc441-bf5c-4207-8b34-0d213130391d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:52.280220 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 160/312] 10.222.0.17 () {56 vars in 1308 bytes} [Sat Sep 10 23:41:52 2022] GET /identity/v3/role_assignments?user.id=31a463346db64e0dabd65403016fbe67&scope.system=all&role.id=900ce80935da4fa59be9334c5c8e0083 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:52.280986 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0dc441-bf5c-4207-8b34-0d213130391d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:52.300788 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:52.316779 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9db304-6564-41e2-a687-cc508fa01bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:52.321342 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9db304-6564-41e2-a687-cc508fa01bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:52.321342 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9db304-6564-41e2-a687-cc508fa01bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:52.321342 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9db304-6564-41e2-a687-cc508fa01bc1 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:52.321342 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9db304-6564-41e2-a687-cc508fa01bc1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:52.321342 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9db304-6564-41e2-a687-cc508fa01bc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:52.321702 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9db304-6564-41e2-a687-cc508fa01bc1 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:52.340688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 153/313] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:41:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:52.342705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9db304-6564-41e2-a687-cc508fa01bc1 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:52.429654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0dc441-bf5c-4207-8b34-0d213130391d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:52.439229 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:52.473305 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9db304-6564-41e2-a687-cc508fa01bc1 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:52.481008 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb670f-127c-4fb5-a3d4-01dc755f6fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:52.481588 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb670f-127c-4fb5-a3d4-01dc755f6fef None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:52.481866 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb670f-127c-4fb5-a3d4-01dc755f6fef None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:52.482135 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb670f-127c-4fb5-a3d4-01dc755f6fef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:52.514532 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329bcba-51df-4979-9c3d-b78065ca49e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:52.515133 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329bcba-51df-4979-9c3d-b78065ca49e9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:52.515413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329bcba-51df-4979-9c3d-b78065ca49e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:52.515683 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329bcba-51df-4979-9c3d-b78065ca49e9 None admin] PATH_INFO: `/v3/users/df486f35f7a540c385f0bc07ccd7cc26` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:52.516971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329bcba-51df-4979-9c3d-b78065ca49e9 None admin] RBAC: Authorizing `identity:get_user(user_id=df486f35f7a540c385f0bc07ccd7cc26)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:52.518527 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329bcba-51df-4979-9c3d-b78065ca49e9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:52.520523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 154/314] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:52 2022] GET /identity/v3/users/df486f35f7a540c385f0bc07ccd7cc26 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:52.525614 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329bcba-51df-4979-9c3d-b78065ca49e9 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:52.570699 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb670f-127c-4fb5-a3d4-01dc755f6fef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:52.573289 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb670f-127c-4fb5-a3d4-01dc755f6fef None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:52.598034 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:52.656033 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329bcba-51df-4979-9c3d-b78065ca49e9 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:52.692660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 161/315] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:41:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:41:52.698109 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39eb930-6217-45ee-a304-c404983c544f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:52.698851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb670f-127c-4fb5-a3d4-01dc755f6fef None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:52.699168 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39eb930-6217-45ee-a304-c404983c544f None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:52.699546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39eb930-6217-45ee-a304-c404983c544f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:52.699915 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39eb930-6217-45ee-a304-c404983c544f None admin] PATH_INFO: `/v3/system/users/df486f35f7a540c385f0bc07ccd7cc26/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:52.701718 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39eb930-6217-45ee-a304-c404983c544f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=df486f35f7a540c385f0bc07ccd7cc26, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:52.703386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39eb930-6217-45ee-a304-c404983c544f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:52.707990 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:52.802202 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb670f-127c-4fb5-a3d4-01dc755f6fef None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:52.803767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 155/316] 10.222.0.17 () {58 vars in 1334 bytes} [Sat Sep 10 23:41:52 2022] PUT /identity/v3/system/users/df486f35f7a540c385f0bc07ccd7cc26/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:41:52.848430 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a821e60-dd0b-4a1f-beaf-f3eaefdf26c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:52.849412 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a821e60-dd0b-4a1f-beaf-f3eaefdf26c5 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:52.849821 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a821e60-dd0b-4a1f-beaf-f3eaefdf26c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:52.850284 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a821e60-dd0b-4a1f-beaf-f3eaefdf26c5 None admin] PATH_INFO: `/v3/projects/15d53acb4e8c43348673ba66109e0371` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:52.851647 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a821e60-dd0b-4a1f-beaf-f3eaefdf26c5 None admin] RBAC: Authorizing `identity:get_project(project_id=15d53acb4e8c43348673ba66109e0371)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:52.853337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a821e60-dd0b-4a1f-beaf-f3eaefdf26c5 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:52.855340 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 162/317] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:52 2022] GET /identity/v3/projects/15d53acb4e8c43348673ba66109e0371 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:41:52.862728 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39eb930-6217-45ee-a304-c404983c544f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:52.986839 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39eb930-6217-45ee-a304-c404983c544f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:53.028837 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:53.040377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb3e04-3236-472b-b93f-7809f2bc16ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:53.040988 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb3e04-3236-472b-b93f-7809f2bc16ea None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:53.041267 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb3e04-3236-472b-b93f-7809f2bc16ea None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:53.041536 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb3e04-3236-472b-b93f-7809f2bc16ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:53.042264 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcb3e04-3236-472b-b93f-7809f2bc16ea None admin] RBAC: Adding query filter params (role.id=900ce80935da4fa59be9334c5c8e0083, scope.project.id=15d53acb4e8c43348673ba66109e0371, user.id=67b2fb0ba6574c42a17448654d297564) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:53.042779 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcb3e04-3236-472b-b93f-7809f2bc16ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:53.044906 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcb3e04-3236-472b-b93f-7809f2bc16ea None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:53.052151 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 156/318] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:41:52 2022] GET /identity/v3/role_assignments?user.id=67b2fb0ba6574c42a17448654d297564&scope.project.id=15d53acb4e8c43348673ba66109e0371&role.id=900ce80935da4fa59be9334c5c8e0083 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:53.923200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4654cd-a2bd-4a11-85dd-5c3b07adc568 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:53.923737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4654cd-a2bd-4a11-85dd-5c3b07adc568 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:53.923737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4654cd-a2bd-4a11-85dd-5c3b07adc568 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:53.925057 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 163/319] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:53.930154 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f98ea0-106a-4393-aebd-af061d6e7790 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:53.930541 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f98ea0-106a-4393-aebd-af061d6e7790 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:53.930722 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f98ea0-106a-4393-aebd-af061d6e7790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:53.949923 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-66f98ea0-106a-4393-aebd-af061d6e7790 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:53.951866 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-66f98ea0-106a-4393-aebd-af061d6e7790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:54.059210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 157/320] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:54.340978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4654cd-a2bd-4a11-85dd-5c3b07adc568 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:54.416124 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:54.442635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4654cd-a2bd-4a11-85dd-5c3b07adc568 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:54.480817 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b05ed72-cfe4-40db-9243-57993e0a7118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:54.482164 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b05ed72-cfe4-40db-9243-57993e0a7118 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:54.482470 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b05ed72-cfe4-40db-9243-57993e0a7118 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:54.482771 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b05ed72-cfe4-40db-9243-57993e0a7118 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:54.483948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b05ed72-cfe4-40db-9243-57993e0a7118 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:54.485420 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b05ed72-cfe4-40db-9243-57993e0a7118 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:54.487265 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 164/321] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:41:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:54.492819 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f98ea0-106a-4393-aebd-af061d6e7790 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:54.589838 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bb7de-749b-4e4c-8def-9141f7387682 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:54.590019 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bb7de-749b-4e4c-8def-9141f7387682 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:54.590294 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bb7de-749b-4e4c-8def-9141f7387682 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:54.592924 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 165/322] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:41:54.593398 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f98ea0-106a-4393-aebd-af061d6e7790 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:54.597886 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4b6a9-f9a5-4c92-9df6-7c81f0617ad5 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:54.598214 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4b6a9-f9a5-4c92-9df6-7c81f0617ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:54.598415 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4b6a9-f9a5-4c92-9df6-7c81f0617ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:54.611145 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:54.617111 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-c3e4b6a9-f9a5-4c92-9df6-7c81f0617ad5 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:54.618345 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c3e4b6a9-f9a5-4c92-9df6-7c81f0617ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:54.631076 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138151b-7231-4872-b2d5-81c00efe1e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:54.631677 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138151b-7231-4872-b2d5-81c00efe1e8e None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:54.631948 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138151b-7231-4872-b2d5-81c00efe1e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:54.632213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138151b-7231-4872-b2d5-81c00efe1e8e None admin] PATH_INFO: `/v3/groups` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:54.632853 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138151b-7231-4872-b2d5-81c00efe1e8e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:54.634311 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138151b-7231-4872-b2d5-81c00efe1e8e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:54.689621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 166/323] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:54.728320 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 158/324] 10.222.0.17 () {60 vars in 1134 bytes} [Sat Sep 10 23:41:54 2022] POST /identity/v3/groups => generated 226 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:41:54.968881 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4b6a9-f9a5-4c92-9df6-7c81f0617ad5 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:55.045457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71313be-c807-46b5-94bc-360b9d8901ac None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:55.045960 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71313be-c807-46b5-94bc-360b9d8901ac None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:55.046031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71313be-c807-46b5-94bc-360b9d8901ac None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:55.048791 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:55.049076 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 159/325] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:55.053333 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c586f4-d060-4d31-8ade-7595e5a35324 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:55.053607 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c586f4-d060-4d31-8ade-7595e5a35324 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:55.053875 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c586f4-d060-4d31-8ade-7595e5a35324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:55.074761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-91c586f4-d060-4d31-8ade-7595e5a35324 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:55.076224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-91c586f4-d060-4d31-8ade-7595e5a35324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:55.076974 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4b6a9-f9a5-4c92-9df6-7c81f0617ad5 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:55.115911 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee5fcb-4a5d-41fd-a3ea-7ef4ed8c9ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:55.116597 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee5fcb-4a5d-41fd-a3ea-7ef4ed8c9ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:55.116875 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee5fcb-4a5d-41fd-a3ea-7ef4ed8c9ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:55.117150 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee5fcb-4a5d-41fd-a3ea-7ef4ed8c9ec5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:55.128063 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee5fcb-4a5d-41fd-a3ea-7ef4ed8c9ec5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:55.132934 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-17ee5fcb-4a5d-41fd-a3ea-7ef4ed8c9ec5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 23:41:55.132934 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:55.132934 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:55.132934 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:55.132934 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:55.132934 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:55.132934 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:55.132934 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:55.132934 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:55.132934 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:55.132934 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:55.132934 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:55.132934 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:55.132934 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:55.132934 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:55.132934 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:55.133797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:55.133797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:55.133797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:55.133797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:55.133797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:55.133797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:55.133797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:55.133797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:55.133797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:55.133797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:55.133797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:55.133797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:55.133797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:55.133797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:55.133797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:55.133797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:55.134567 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:55.134567 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:55.134567 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:55.134567 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:55.134567 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:55.134567 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:55.134567 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:55.134567 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:55.134567 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:55.134567 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:55.134567 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 23:41:55.134567 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:41:55.134567 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 167/326] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:55.140646 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee5fcb-4a5d-41fd-a3ea-7ef4ed8c9ec5 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:55.167596 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 160/327] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 23:41:55.253814 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee5fcb-4a5d-41fd-a3ea-7ef4ed8c9ec5 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:55.285768 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:55.290873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae6ee6-0a98-4957-b777-57eb0f788a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:55.291658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae6ee6-0a98-4957-b777-57eb0f788a20 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:55.292092 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae6ee6-0a98-4957-b777-57eb0f788a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:55.292575 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae6ee6-0a98-4957-b777-57eb0f788a20 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:55.293457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae6ee6-0a98-4957-b777-57eb0f788a20 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:55.299034 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae6ee6-0a98-4957-b777-57eb0f788a20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:55.300489 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae6ee6-0a98-4957-b777-57eb0f788a20 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:55.310535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 168/328] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:41:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:41:55.319687 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c586f4-d060-4d31-8ade-7595e5a35324 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:55.438322 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c586f4-d060-4d31-8ade-7595e5a35324 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:55.446245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:55.451770 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae6ee6-0a98-4957-b777-57eb0f788a20 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:55.492652 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ede625-b0c3-46aa-b7e3-58ae179c6023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:55.493340 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ede625-b0c3-46aa-b7e3-58ae179c6023 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:55.493617 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ede625-b0c3-46aa-b7e3-58ae179c6023 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:55.493945 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ede625-b0c3-46aa-b7e3-58ae179c6023 None admin] PATH_INFO: `/v3/users/31a463346db64e0dabd65403016fbe67` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:55.498709 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ede625-b0c3-46aa-b7e3-58ae179c6023 None admin] RBAC: Authorizing `identity:get_user(user_id=31a463346db64e0dabd65403016fbe67)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:55.499952 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ede625-b0c3-46aa-b7e3-58ae179c6023 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:55.501771 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 161/329] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:55 2022] GET /identity/v3/users/31a463346db64e0dabd65403016fbe67 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:41:55.510072 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ede625-b0c3-46aa-b7e3-58ae179c6023 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:55.607588 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae6ee6-0a98-4957-b777-57eb0f788a20 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:55.647122 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ede625-b0c3-46aa-b7e3-58ae179c6023 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:55.648846 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2b16d-8dd6-4770-95e4-632d325f6941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:55.650623 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2b16d-8dd6-4770-95e4-632d325f6941 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:55.651163 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2b16d-8dd6-4770-95e4-632d325f6941 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:55.651631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2b16d-8dd6-4770-95e4-632d325f6941 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:55.666263 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e2b16d-8dd6-4770-95e4-632d325f6941 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:55.673952 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-36e2b16d-8dd6-4770-95e4-632d325f6941 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:55.673952 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:55.673952 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:55.673952 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:55.673952 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:55.673952 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:55.673952 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:55.673952 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:55.673952 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:55.673952 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:55.673952 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:55.673952 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:55.673952 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:55.673952 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:55.673952 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:55.673952 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:55.674902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:55.674902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:55.674902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:55.674902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:55.674902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:55.674902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:55.674902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:55.674902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:55.674902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:55.674902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:55.674902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:55.674902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:55.674902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:55.674902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:55.674902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:55.674902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:55.675690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:55.675690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:55.675690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:55.675690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:55.675690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:55.675690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:55.675690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:55.675690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:55.675690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:55.675690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:55.675690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:55.675690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:41:55.676291 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 169/330] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:55.683114 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2b16d-8dd6-4770-95e4-632d325f6941 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:55.696190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6033728-898a-42f9-afd5-2ce0126a9249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:55.696840 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6033728-898a-42f9-afd5-2ce0126a9249 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:55.697519 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6033728-898a-42f9-afd5-2ce0126a9249 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:55.697804 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6033728-898a-42f9-afd5-2ce0126a9249 None admin] PATH_INFO: `/v3/system/users/31a463346db64e0dabd65403016fbe67/roles/900ce80935da4fa59be9334c5c8e0083` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:55.700447 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6033728-898a-42f9-afd5-2ce0126a9249 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=31a463346db64e0dabd65403016fbe67, role_id=900ce80935da4fa59be9334c5c8e0083)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:55.701825 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6033728-898a-42f9-afd5-2ce0126a9249 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:55.752784 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:55.765603 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 162/331] 10.222.0.17 () {58 vars in 1334 bytes} [Sat Sep 10 23:41:55 2022] PUT /identity/v3/system/users/31a463346db64e0dabd65403016fbe67/roles/900ce80935da4fa59be9334c5c8e0083 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:41:55.827996 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2b16d-8dd6-4770-95e4-632d325f6941 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:55.882715 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff0440-ea8d-416c-ab36-a03caa9d3fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:55.883346 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff0440-ea8d-416c-ab36-a03caa9d3fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:55.883626 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff0440-ea8d-416c-ab36-a03caa9d3fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:55.883898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff0440-ea8d-416c-ab36-a03caa9d3fe9 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:55.884589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff0440-ea8d-416c-ab36-a03caa9d3fe9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:55.884889 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff0440-ea8d-416c-ab36-a03caa9d3fe9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:55.886292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff0440-ea8d-416c-ab36-a03caa9d3fe9 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:55.898508 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 170/332] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:41:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:41:55.905070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6033728-898a-42f9-afd5-2ce0126a9249 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:55.972256 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:56.041988 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6033728-898a-42f9-afd5-2ce0126a9249 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:56.080838 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd7624-7673-427d-a04e-331173c25d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:56.081566 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd7624-7673-427d-a04e-331173c25d38 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:56.081787 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd7624-7673-427d-a04e-331173c25d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:56.081989 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd7624-7673-427d-a04e-331173c25d38 None admin] PATH_INFO: `/v3/users/df486f35f7a540c385f0bc07ccd7cc26` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:56.083160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facd7624-7673-427d-a04e-331173c25d38 None admin] RBAC: Authorizing `identity:get_user(user_id=df486f35f7a540c385f0bc07ccd7cc26)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:56.084602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facd7624-7673-427d-a04e-331173c25d38 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:56.086472 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 163/333] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:41:55 2022] GET /identity/v3/users/df486f35f7a540c385f0bc07ccd7cc26 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:56.092908 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff0440-ea8d-416c-ab36-a03caa9d3fe9 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:56.215094 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff0440-ea8d-416c-ab36-a03caa9d3fe9 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:56.242335 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:56.267526 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd7c2d-e2e5-4c63-b5ab-2d2efb1795d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:56.268146 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd7c2d-e2e5-4c63-b5ab-2d2efb1795d8 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:56.268423 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd7c2d-e2e5-4c63-b5ab-2d2efb1795d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:56.268698 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd7c2d-e2e5-4c63-b5ab-2d2efb1795d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:56.269426 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd7c2d-e2e5-4c63-b5ab-2d2efb1795d8 None admin] RBAC: Adding query filter params (role.id=824f5fe1f02b447382190a8c236c27db, user.id=df486f35f7a540c385f0bc07ccd7cc26, scope.system=all) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:56.269732 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd7c2d-e2e5-4c63-b5ab-2d2efb1795d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:56.271201 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd7c2d-e2e5-4c63-b5ab-2d2efb1795d8 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:56.285170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 171/334] 10.222.0.17 () {56 vars in 1308 bytes} [Sat Sep 10 23:41:56 2022] GET /identity/v3/role_assignments?user.id=df486f35f7a540c385f0bc07ccd7cc26&scope.system=all&role.id=824f5fe1f02b447382190a8c236c27db => generated 510 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:41:57.673730 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ed2b2-695f-4dce-ac88-03727c7e7a65 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:57.674775 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ed2b2-695f-4dce-ac88-03727c7e7a65 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:57.675276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ed2b2-695f-4dce-ac88-03727c7e7a65 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:57.677067 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 164/335] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:57.683026 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2551b-d656-4c9e-92af-2a257e171794 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:57.683867 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2551b-d656-4c9e-92af-2a257e171794 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:57.684362 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2551b-d656-4c9e-92af-2a257e171794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:57.711723 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-ecf2551b-d656-4c9e-92af-2a257e171794 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:57.713222 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ecf2551b-d656-4c9e-92af-2a257e171794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:57.792063 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 172/336] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:58.241575 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ed2b2-695f-4dce-ac88-03727c7e7a65 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:58.279840 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e875e2-cdad-4d0b-a3dd-626e2aa1af37 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:58.280789 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e875e2-cdad-4d0b-a3dd-626e2aa1af37 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:58.281483 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e875e2-cdad-4d0b-a3dd-626e2aa1af37 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:58.283977 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 173/337] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:58.291879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcac114-464e-4d40-a201-ac97689436a4 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:58.292449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcac114-464e-4d40-a201-ac97689436a4 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:58.292944 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcac114-464e-4d40-a201-ac97689436a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:58.334929 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-5fcac114-464e-4d40-a201-ac97689436a4 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:58.336659 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5fcac114-464e-4d40-a201-ac97689436a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:58.463790 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ed2b2-695f-4dce-ac88-03727c7e7a65 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:58.478376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:58.485021 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 174/338] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 23:41:58.540046 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43aa64f-2b63-47b8-842a-da1675ec79ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:58.540736 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43aa64f-2b63-47b8-842a-da1675ec79ac None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:58.541127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43aa64f-2b63-47b8-842a-da1675ec79ac None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:58.541487 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43aa64f-2b63-47b8-842a-da1675ec79ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:58.554292 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43aa64f-2b63-47b8-842a-da1675ec79ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:58.559980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-a43aa64f-2b63-47b8-842a-da1675ec79ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:58.559980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:58.559980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:58.559980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:58.559980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:58.559980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:58.559980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:58.559980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:58.559980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:58.559980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:58.559980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:58.559980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:58.559980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:58.559980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:58.559980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:58.559980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:58.560981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:58.560981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:58.560981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:58.560981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:58.560981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:58.560981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:58.560981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:58.560981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:58.560981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:58.560981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:58.560981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:58.560981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:58.560981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:58.560981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:58.560981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:58.560981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:58.561784 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:58.561784 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:58.561784 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:58.561784 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:58.561784 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:58.561784 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:58.561784 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:58.561784 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:58.561784 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:58.561784 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:58.561784 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:58.561784 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:41:58.561784 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 165/339] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:58.567027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcac114-464e-4d40-a201-ac97689436a4 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:58.620257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:58.663595 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de129518-2499-4ae8-ad00-e902fbb3d733 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:58.664205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de129518-2499-4ae8-ad00-e902fbb3d733 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:58.664671 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de129518-2499-4ae8-ad00-e902fbb3d733 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:58.666250 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 166/340] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:58.672356 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e74045-d64c-41ab-91af-97c85dd1e6f3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:58.672933 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e74045-d64c-41ab-91af-97c85dd1e6f3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:58.673395 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e74045-d64c-41ab-91af-97c85dd1e6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:58.703640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-e9e74045-d64c-41ab-91af-97c85dd1e6f3 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:58.705664 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e9e74045-d64c-41ab-91af-97c85dd1e6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:58.714603 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcac114-464e-4d40-a201-ac97689436a4 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:58.772159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eabd81-3e77-4d24-a38c-bcb6713bc04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:58.772772 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eabd81-3e77-4d24-a38c-bcb6713bc04b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:58.773044 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eabd81-3e77-4d24-a38c-bcb6713bc04b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:58.773312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eabd81-3e77-4d24-a38c-bcb6713bc04b None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:58.773975 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eabd81-3e77-4d24-a38c-bcb6713bc04b None admin] RBAC: Adding query filter params (name=member) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:58.774270 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eabd81-3e77-4d24-a38c-bcb6713bc04b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:58.775677 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eabd81-3e77-4d24-a38c-bcb6713bc04b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:58.797131 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 175/341] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:41:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:58.802990 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eabd81-3e77-4d24-a38c-bcb6713bc04b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:58.844011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 167/342] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:58.913337 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e74045-d64c-41ab-91af-97c85dd1e6f3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:58.967275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:58.973605 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eabd81-3e77-4d24-a38c-bcb6713bc04b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:59.050289 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5235af7-dec7-4f51-87dc-ae66989feb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:59.051070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5235af7-dec7-4f51-87dc-ae66989feb5d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:59.051352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5235af7-dec7-4f51-87dc-ae66989feb5d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:59.051616 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5235af7-dec7-4f51-87dc-ae66989feb5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:59.086757 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:59.093823 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e74045-d64c-41ab-91af-97c85dd1e6f3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:59.142963 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc6ca5-36d3-47a7-9184-316789c523d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:59.143999 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc6ca5-36d3-47a7-9184-316789c523d0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:59.145153 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc6ca5-36d3-47a7-9184-316789c523d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:59.145626 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc6ca5-36d3-47a7-9184-316789c523d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:59.158125 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5235af7-dec7-4f51-87dc-ae66989feb5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:59.160693 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5235af7-dec7-4f51-87dc-ae66989feb5d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:59.166028 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc6ca5-36d3-47a7-9184-316789c523d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:59.176767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-57bc6ca5-36d3-47a7-9184-316789c523d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:59.176767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:59.176767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:59.176767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:59.176767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:59.176767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:59.176767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:59.176767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:59.176767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:59.176767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:59.176767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:59.176767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:59.176767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:59.176767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:59.176767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:59.176767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:59.177690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:59.177690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:59.177690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:59.177690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:59.177690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:59.177690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:59.177690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:59.177690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:59.177690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:59.177690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:59.177690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:59.177690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:59.177690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:59.177690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:59.177690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:59.177690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:59.178466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:59.178466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:59.178466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:59.178466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:59.178466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:59.178466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:59.178466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:59.178466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:59.178466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:59.178466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:59.178466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:41:59.178466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:41:59.180353 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 168/343] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:41:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:41:59.183903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a730777-8fd8-463d-a824-eaa0f853baa5 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:59.184189 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a730777-8fd8-463d-a824-eaa0f853baa5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:59.185002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a730777-8fd8-463d-a824-eaa0f853baa5 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:59.186320 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 169/344] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:59.195605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a730777-8fd8-463d-a824-eaa0f853baa5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:59.278703 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:59.285897 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 176/345] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:41:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 10 23:41:59.292063 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cedfd-985c-44de-b933-211ec7aff07f None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:59.292338 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cedfd-985c-44de-b933-211ec7aff07f None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:59.292606 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cedfd-985c-44de-b933-211ec7aff07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:59.319806 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-ca4cedfd-985c-44de-b933-211ec7aff07f None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:59.321797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ca4cedfd-985c-44de-b933-211ec7aff07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:59.387124 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a730777-8fd8-463d-a824-eaa0f853baa5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:59.401893 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 177/346] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:59.403748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4cedfd-985c-44de-b933-211ec7aff07f None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:59.452804 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad0cad-bcc3-457b-94d3-ddf02ff71001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:59.453733 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad0cad-bcc3-457b-94d3-ddf02ff71001 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:59.454144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad0cad-bcc3-457b-94d3-ddf02ff71001 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:59.454547 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad0cad-bcc3-457b-94d3-ddf02ff71001 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:59.455348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad0cad-bcc3-457b-94d3-ddf02ff71001 None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:59.455781 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad0cad-bcc3-457b-94d3-ddf02ff71001 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:59.457304 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad0cad-bcc3-457b-94d3-ddf02ff71001 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:59.477062 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 170/347] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:41:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:59.478193 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad0cad-bcc3-457b-94d3-ddf02ff71001 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:59.516332 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:59.522449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4cedfd-985c-44de-b933-211ec7aff07f None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:59.556153 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:59.566945 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff9691-9c79-4c6c-b6e5-ae0e73fbac15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:59.567511 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff9691-9c79-4c6c-b6e5-ae0e73fbac15 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:59.568233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff9691-9c79-4c6c-b6e5-ae0e73fbac15 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:59.568675 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff9691-9c79-4c6c-b6e5-ae0e73fbac15 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:59.580673 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ff9691-9c79-4c6c-b6e5-ae0e73fbac15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:41:59.587253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-a4ff9691-9c79-4c6c-b6e5-ae0e73fbac15 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 23:41:59.587253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:41:59.587253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:41:59.587253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:41:59.587253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:41:59.587253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:41:59.587253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:41:59.587253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:41:59.587253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:41:59.587253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:41:59.587253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:41:59.587253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:41:59.587253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:41:59.587253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:41:59.587253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:41:59.587253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:41:59.588333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:41:59.588333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:41:59.588333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:41:59.588333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:41:59.588333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:41:59.588333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:41:59.588333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:41:59.588333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:41:59.588333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:41:59.588333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:41:59.588333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:41:59.588333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:41:59.588333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:41:59.588333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:41:59.588333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:41:59.588333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:41:59.589326 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:41:59.589326 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:41:59.589326 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:41:59.589326 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:41:59.589326 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:41:59.589326 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:41:59.589326 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:41:59.589326 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:41:59.589326 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:41:59.589326 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:41:59.589326 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 23:41:59.589326 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:41:59.591839 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 178/348] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Sep 10 23:41:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:41:59.592842 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff9691-9c79-4c6c-b6e5-ae0e73fbac15 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:59.623323 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad0cad-bcc3-457b-94d3-ddf02ff71001 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:59.658693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146218b-c7e8-4d4b-81f3-a741c6a32a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:59.659873 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146218b-c7e8-4d4b-81f3-a741c6a32a52 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:59.660286 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146218b-c7e8-4d4b-81f3-a741c6a32a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:59.660719 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146218b-c7e8-4d4b-81f3-a741c6a32a52 None admin] PATH_INFO: `/v3/projects/59ae2524fa884ab7a9682a93d368b500` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:59.662248 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146218b-c7e8-4d4b-81f3-a741c6a32a52 None admin] RBAC: Authorizing `identity:get_project(project_id=59ae2524fa884ab7a9682a93d368b500)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:59.664345 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146218b-c7e8-4d4b-81f3-a741c6a32a52 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:59.666557 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 171/349] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:41:59 2022] GET /identity/v3/projects/59ae2524fa884ab7a9682a93d368b500 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:59.668297 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbb8c3-c168-43da-9975-2349852c7f21 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:59.669397 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbb8c3-c168-43da-9975-2349852c7f21 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:59.670149 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbb8c3-c168-43da-9975-2349852c7f21 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:59.671818 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 172/350] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:41:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:41:59.672709 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbb8c3-c168-43da-9975-2349852c7f21 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:41:59.724744 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff9691-9c79-4c6c-b6e5-ae0e73fbac15 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:59.785598 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:59.786845 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c1af4-c217-46e7-8675-3a580c20b066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:59.787462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c1af4-c217-46e7-8675-3a580c20b066 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:59.787734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c1af4-c217-46e7-8675-3a580c20b066 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:59.787999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c1af4-c217-46e7-8675-3a580c20b066 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:59.805098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbb8c3-c168-43da-9975-2349852c7f21 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:41:59.850323 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae6858-6be6-44a1-b457-553545a172d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:41:59.850878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae6858-6be6-44a1-b457-553545a172d3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:59.851150 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae6858-6be6-44a1-b457-553545a172d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:59.851421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae6858-6be6-44a1-b457-553545a172d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:59.852082 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae6858-6be6-44a1-b457-553545a172d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:41:59.852379 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae6858-6be6-44a1-b457-553545a172d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:59.853788 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae6858-6be6-44a1-b457-553545a172d3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:59.859937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:41:59.865962 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 173/351] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:41:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:41:59.867225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bbc47-7ae8-434b-b260-de7a20a897c4 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:41:59.872068 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bbc47-7ae8-434b-b260-de7a20a897c4 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:41:59.872068 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bbc47-7ae8-434b-b260-de7a20a897c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:41:59.875897 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c1af4-c217-46e7-8675-3a580c20b066 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:41:59.878593 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c1af4-c217-46e7-8675-3a580c20b066 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:41:59.891893 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-829bbc47-7ae8-434b-b260-de7a20a897c4 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:41:59.893403 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-829bbc47-7ae8-434b-b260-de7a20a897c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:41:59.986176 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 174/352] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:41:59.987446 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829bbc47-7ae8-434b-b260-de7a20a897c4 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:00.024790 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 179/353] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:41:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:42:00.026565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c1af4-c217-46e7-8675-3a580c20b066 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:00.122554 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:00.144135 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:00.152254 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829bbc47-7ae8-434b-b260-de7a20a897c4 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:00.164741 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c1af4-c217-46e7-8675-3a580c20b066 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:00.203255 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfca6cf-f22b-45e3-b513-1ec9a48d94f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:00.204105 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfca6cf-f22b-45e3-b513-1ec9a48d94f2 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:00.204482 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfca6cf-f22b-45e3-b513-1ec9a48d94f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:00.205072 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfca6cf-f22b-45e3-b513-1ec9a48d94f2 None admin] PATH_INFO: `/v3/groups/e943de99a6e9465cb060d45671dc02e6` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:00.217316 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfca6cf-f22b-45e3-b513-1ec9a48d94f2 None admin] RBAC: Authorizing `identity:get_group(group_id=e943de99a6e9465cb060d45671dc02e6)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:00.219436 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfca6cf-f22b-45e3-b513-1ec9a48d94f2 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:00.221640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 175/354] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Sep 10 23:41:59 2022] GET /identity/v3/groups/e943de99a6e9465cb060d45671dc02e6 => generated 226 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:00.222889 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfca6cf-f22b-45e3-b513-1ec9a48d94f2 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:00.224118 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2836e-4b95-49c3-9e9f-9530290e530f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:00.225042 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2836e-4b95-49c3-9e9f-9530290e530f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:00.225417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2836e-4b95-49c3-9e9f-9530290e530f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:00.225787 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2836e-4b95-49c3-9e9f-9530290e530f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:00.241547 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b2836e-4b95-49c3-9e9f-9530290e530f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:00.247915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-01b2836e-4b95-49c3-9e9f-9530290e530f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 23:42:00.247915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:00.247915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:00.247915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:00.247915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:00.247915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:00.247915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:00.247915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:00.247915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:00.247915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:00.247915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:00.247915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:00.247915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:00.247915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:00.247915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:00.247915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:00.248808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:00.248808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:00.248808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:00.248808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:00.248808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:00.248808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:00.248808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:00.248808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:00.248808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:00.248808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:00.248808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:00.248808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:00.248808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:00.248808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:00.248808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:00.248808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:00.249750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:00.249750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:00.249750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:00.249750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:00.249750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:00.249750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:00.249750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:00.249750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:00.249750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:00.249750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:00.249750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 23:42:00.249750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:00.249750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 180/355] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:42:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:00.251348 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2836e-4b95-49c3-9e9f-9530290e530f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:00.353487 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfca6cf-f22b-45e3-b513-1ec9a48d94f2 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:00.377329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2836e-4b95-49c3-9e9f-9530290e530f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:00.400623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101885e-ec6b-4c33-9955-b5dd7bcd04a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:00.401228 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101885e-ec6b-4c33-9955-b5dd7bcd04a6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:00.401497 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101885e-ec6b-4c33-9955-b5dd7bcd04a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:00.401759 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101885e-ec6b-4c33-9955-b5dd7bcd04a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:00.416317 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:00.421417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934d77b-7133-462e-bc7e-2c153526dcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:00.422689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934d77b-7133-462e-bc7e-2c153526dcae None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:00.422689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934d77b-7133-462e-bc7e-2c153526dcae None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:00.422689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934d77b-7133-462e-bc7e-2c153526dcae None admin] PATH_INFO: `/v3/projects/15d53acb4e8c43348673ba66109e0371` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:00.423790 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934d77b-7133-462e-bc7e-2c153526dcae None admin] RBAC: Authorizing `identity:get_project(project_id=15d53acb4e8c43348673ba66109e0371)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:00.425218 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934d77b-7133-462e-bc7e-2c153526dcae None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:00.427223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 181/356] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:42:00 2022] GET /identity/v3/projects/15d53acb4e8c43348673ba66109e0371 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:00.427836 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934d77b-7133-462e-bc7e-2c153526dcae None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:00.450644 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:00.484550 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3101885e-ec6b-4c33-9955-b5dd7bcd04a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:00.486030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3101885e-ec6b-4c33-9955-b5dd7bcd04a6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:00.554068 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934d77b-7133-462e-bc7e-2c153526dcae None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:00.596359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25b7d2-98e5-4685-9e2b-b2b81fd8eb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:00.597328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25b7d2-98e5-4685-9e2b-b2b81fd8eb99 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:00.597662 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25b7d2-98e5-4685-9e2b-b2b81fd8eb99 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:00.597992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25b7d2-98e5-4685-9e2b-b2b81fd8eb99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:00.598831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25b7d2-98e5-4685-9e2b-b2b81fd8eb99 None admin] RBAC: Adding query filter params (group.id=e943de99a6e9465cb060d45671dc02e6, role.id=824f5fe1f02b447382190a8c236c27db, scope.project.id=59ae2524fa884ab7a9682a93d368b500) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:00.599132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25b7d2-98e5-4685-9e2b-b2b81fd8eb99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:00.600556 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25b7d2-98e5-4685-9e2b-b2b81fd8eb99 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:00.607298 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 182/357] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Sep 10 23:42:00 2022] GET /identity/v3/role_assignments?group.id=e943de99a6e9465cb060d45671dc02e6&scope.project.id=59ae2524fa884ab7a9682a93d368b500&role.id=824f5fe1f02b447382190a8c236c27db => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:00.607554 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25b7d2-98e5-4685-9e2b-b2b81fd8eb99 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:00.621496 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 176/358] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:42:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:42:00.621496 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101885e-ec6b-4c33-9955-b5dd7bcd04a6 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:00.678111 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:00.720259 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:00.753233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25b7d2-98e5-4685-9e2b-b2b81fd8eb99 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:00.771751 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101885e-ec6b-4c33-9955-b5dd7bcd04a6 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:00.800750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2633cb6d-178a-42bd-87da-ac4dca4cb91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:00.802134 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633cb6d-178a-42bd-87da-ac4dca4cb91f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:00.802554 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633cb6d-178a-42bd-87da-ac4dca4cb91f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:00.803027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633cb6d-178a-42bd-87da-ac4dca4cb91f None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:00.804110 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2633cb6d-178a-42bd-87da-ac4dca4cb91f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:00.804549 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2633cb6d-178a-42bd-87da-ac4dca4cb91f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:00.806604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2633cb6d-178a-42bd-87da-ac4dca4cb91f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:00.818313 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 183/359] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:42:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:00.819923 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54671961-63fa-4b6a-ab55-0566f93be7c2 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:00.820205 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54671961-63fa-4b6a-ab55-0566f93be7c2 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:00.820471 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54671961-63fa-4b6a-ab55-0566f93be7c2 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:00.821815 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 184/360] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:00.822474 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54671961-63fa-4b6a-ab55-0566f93be7c2 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:00.827153 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027d5c3-1528-4a9f-abcc-88859eaf77f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:00.829339 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027d5c3-1528-4a9f-abcc-88859eaf77f1 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:00.829617 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027d5c3-1528-4a9f-abcc-88859eaf77f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:00.829886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027d5c3-1528-4a9f-abcc-88859eaf77f1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:00.849787 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d027d5c3-1528-4a9f-abcc-88859eaf77f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:00.854588 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-d027d5c3-1528-4a9f-abcc-88859eaf77f1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 23:42:00.854588 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:00.854588 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:00.854588 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:00.854588 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:00.854588 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:00.854588 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:00.854588 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:00.854588 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:00.854588 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:00.854588 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:00.854588 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:00.854588 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:00.854588 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:00.854588 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:00.854588 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:00.855700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:00.855700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:00.855700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:00.855700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:00.855700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:00.855700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:00.855700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:00.855700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:00.855700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:00.855700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:00.855700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:00.855700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:00.855700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:00.855700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:00.855700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:00.855700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:00.856718 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:00.856718 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:00.856718 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:00.856718 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:00.856718 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:00.856718 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:00.856718 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:00.856718 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:00.856718 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:00.856718 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:00.856718 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 23:42:00.856718 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:00.857606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 177/361] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Sep 10 23:42:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:00.857800 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027d5c3-1528-4a9f-abcc-88859eaf77f1 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:00.972087 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54671961-63fa-4b6a-ab55-0566f93be7c2 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:00.986640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027d5c3-1528-4a9f-abcc-88859eaf77f1 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:00.988690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:01.015652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19814a8e-a2bf-4902-8573-3537f5a20d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:01.016777 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19814a8e-a2bf-4902-8573-3537f5a20d40 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:01.017053 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19814a8e-a2bf-4902-8573-3537f5a20d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:01.017638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19814a8e-a2bf-4902-8573-3537f5a20d40 None admin] PATH_INFO: `/v3/groups/e943de99a6e9465cb060d45671dc02e6` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:01.019662 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19814a8e-a2bf-4902-8573-3537f5a20d40 None admin] RBAC: Authorizing `identity:get_group(group_id=e943de99a6e9465cb060d45671dc02e6)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:01.021767 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19814a8e-a2bf-4902-8573-3537f5a20d40 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:01.024285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 185/362] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Sep 10 23:42:00 2022] GET /identity/v3/groups/e943de99a6e9465cb060d45671dc02e6 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:01.025237 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19814a8e-a2bf-4902-8573-3537f5a20d40 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:01.026215 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd588e54-f857-408e-835e-1299aa6f00af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:01.026690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd588e54-f857-408e-835e-1299aa6f00af None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:01.031542 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd588e54-f857-408e-835e-1299aa6f00af None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:01.031542 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd588e54-f857-408e-835e-1299aa6f00af None admin] PATH_INFO: `/v3/projects/59ae2524fa884ab7a9682a93d368b500` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:01.032619 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd588e54-f857-408e-835e-1299aa6f00af None admin] RBAC: Authorizing `identity:get_project(project_id=59ae2524fa884ab7a9682a93d368b500)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:01.034098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd588e54-f857-408e-835e-1299aa6f00af None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:01.035780 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 178/363] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:42:00 2022] GET /identity/v3/projects/59ae2524fa884ab7a9682a93d368b500 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:01.037033 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8768a-7507-49b5-a720-05dcd7150054 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:01.037310 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8768a-7507-49b5-a720-05dcd7150054 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:01.037578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8768a-7507-49b5-a720-05dcd7150054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:01.040560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:01.061707 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-37e8768a-7507-49b5-a720-05dcd7150054 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:01.063018 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-37e8768a-7507-49b5-a720-05dcd7150054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:01.151983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 179/364] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:01.152848 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e8768a-7507-49b5-a720-05dcd7150054 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:01.156761 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19814a8e-a2bf-4902-8573-3537f5a20d40 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:01.193598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cf253-688b-407b-8f0f-59a57ce979ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:01.194278 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cf253-688b-407b-8f0f-59a57ce979ee None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:01.194585 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cf253-688b-407b-8f0f-59a57ce979ee None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:01.194848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cf253-688b-407b-8f0f-59a57ce979ee None admin] PATH_INFO: `/v3/users/31a463346db64e0dabd65403016fbe67` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:01.196866 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1cf253-688b-407b-8f0f-59a57ce979ee None admin] RBAC: Authorizing `identity:get_user(user_id=31a463346db64e0dabd65403016fbe67)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:01.198295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1cf253-688b-407b-8f0f-59a57ce979ee None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:01.199940 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 186/365] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:42:01 2022] GET /identity/v3/users/31a463346db64e0dabd65403016fbe67 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:01.200960 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cf253-688b-407b-8f0f-59a57ce979ee None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:01.276441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:01.279787 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e8768a-7507-49b5-a720-05dcd7150054 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:01.302254 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:01.332336 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cf253-688b-407b-8f0f-59a57ce979ee None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:01.332559 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f8aba-7d8b-4ec7-a5a2-cae34c2b6e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:01.333458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f8aba-7d8b-4ec7-a5a2-cae34c2b6e50 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:01.333734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f8aba-7d8b-4ec7-a5a2-cae34c2b6e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:01.334006 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f8aba-7d8b-4ec7-a5a2-cae34c2b6e50 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:01.334708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f8aba-7d8b-4ec7-a5a2-cae34c2b6e50 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:01.335011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f8aba-7d8b-4ec7-a5a2-cae34c2b6e50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:01.338029 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f8aba-7d8b-4ec7-a5a2-cae34c2b6e50 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:01.354688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 180/366] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:01.354688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f8aba-7d8b-4ec7-a5a2-cae34c2b6e50 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:01.391126 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a94ea8a-5cd3-4f23-b4be-f8d0d3acdb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:01.391814 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94ea8a-5cd3-4f23-b4be-f8d0d3acdb6d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:01.391992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94ea8a-5cd3-4f23-b4be-f8d0d3acdb6d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:01.392257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94ea8a-5cd3-4f23-b4be-f8d0d3acdb6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:01.393033 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a94ea8a-5cd3-4f23-b4be-f8d0d3acdb6d None admin] RBAC: Adding query filter params (group.id=e943de99a6e9465cb060d45671dc02e6, role.id=824f5fe1f02b447382190a8c236c27db, scope.project.id=15d53acb4e8c43348673ba66109e0371) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:01.393340 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a94ea8a-5cd3-4f23-b4be-f8d0d3acdb6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:01.395279 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a94ea8a-5cd3-4f23-b4be-f8d0d3acdb6d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:01.404000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 187/367] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Sep 10 23:42:01 2022] GET /identity/v3/role_assignments?group.id=e943de99a6e9465cb060d45671dc02e6&scope.project.id=15d53acb4e8c43348673ba66109e0371&role.id=824f5fe1f02b447382190a8c236c27db => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:01.405297 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a94ea8a-5cd3-4f23-b4be-f8d0d3acdb6d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:01.495745 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f8aba-7d8b-4ec7-a5a2-cae34c2b6e50 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:01.550369 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85c7aa-53da-421e-903f-5752f831d9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:01.551217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85c7aa-53da-421e-903f-5752f831d9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:01.551430 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85c7aa-53da-421e-903f-5752f831d9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:01.551708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85c7aa-53da-421e-903f-5752f831d9d4 None admin] PATH_INFO: `/v3/groups/e943de99a6e9465cb060d45671dc02e6` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:01.554307 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85c7aa-53da-421e-903f-5752f831d9d4 None admin] RBAC: Authorizing `identity:get_group(group_id=e943de99a6e9465cb060d45671dc02e6)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:01.556014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85c7aa-53da-421e-903f-5752f831d9d4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:01.558477 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 181/368] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Sep 10 23:42:01 2022] GET /identity/v3/groups/e943de99a6e9465cb060d45671dc02e6 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:01.562826 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85c7aa-53da-421e-903f-5752f831d9d4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:01.569869 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a94ea8a-5cd3-4f23-b4be-f8d0d3acdb6d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:01.606687 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:01.611211 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008071cc-45ba-48f5-8ecb-38e2f54d4a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:01.612880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008071cc-45ba-48f5-8ecb-38e2f54d4a98 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:01.612880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008071cc-45ba-48f5-8ecb-38e2f54d4a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:01.612880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008071cc-45ba-48f5-8ecb-38e2f54d4a98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:01.613324 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008071cc-45ba-48f5-8ecb-38e2f54d4a98 None admin] RBAC: Adding query filter params (role.id=900ce80935da4fa59be9334c5c8e0083, user.id=31a463346db64e0dabd65403016fbe67, scope.system=all) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:01.613635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008071cc-45ba-48f5-8ecb-38e2f54d4a98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:01.615167 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008071cc-45ba-48f5-8ecb-38e2f54d4a98 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:01.627621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:01.629916 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 188/369] 10.222.0.17 () {56 vars in 1308 bytes} [Sat Sep 10 23:42:01 2022] GET /identity/v3/role_assignments?user.id=31a463346db64e0dabd65403016fbe67&scope.system=all&role.id=900ce80935da4fa59be9334c5c8e0083 => generated 510 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:01.634735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008071cc-45ba-48f5-8ecb-38e2f54d4a98 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:01.728469 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85c7aa-53da-421e-903f-5752f831d9d4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:01.771308 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008071cc-45ba-48f5-8ecb-38e2f54d4a98 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:01.781566 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b35d1-b754-4f8c-9c5d-644243ad095e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:01.782540 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b35d1-b754-4f8c-9c5d-644243ad095e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:01.783048 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b35d1-b754-4f8c-9c5d-644243ad095e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:01.783518 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b35d1-b754-4f8c-9c5d-644243ad095e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:01.820741 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b51061-6128-405b-877c-bc1163c11f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:01.821316 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b51061-6128-405b-877c-bc1163c11f04 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:01.821592 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b51061-6128-405b-877c-bc1163c11f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:01.821869 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b51061-6128-405b-877c-bc1163c11f04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:01.836180 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b51061-6128-405b-877c-bc1163c11f04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:01.842847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-87b51061-6128-405b-877c-bc1163c11f04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:42:01.842847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:01.842847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:01.842847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:01.842847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:01.842847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:01.842847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:01.842847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:01.842847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:01.842847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:01.842847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:01.842847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:01.842847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:01.842847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:01.842847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:01.842847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:01.843827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:01.843827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:01.843827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:01.843827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:01.843827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:01.843827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:01.843827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:01.843827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:01.843827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:01.843827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:01.843827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:01.843827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:01.843827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:01.843827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:01.843827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:01.843827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:01.844621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:01.844621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:01.844621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:01.844621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:01.844621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:01.844621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:01.844621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:01.844621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:01.844621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:01.844621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:01.844621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:42:01.844621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:01.844621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 189/370] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:42:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:01.845281 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b51061-6128-405b-877c-bc1163c11f04 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:01.866131 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b35d1-b754-4f8c-9c5d-644243ad095e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:01.868481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b35d1-b754-4f8c-9c5d-644243ad095e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:01.902278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:01.915591 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:01.967587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b51061-6128-405b-877c-bc1163c11f04 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:01.991409 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 182/371] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:42:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:42:01.992717 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b35d1-b754-4f8c-9c5d-644243ad095e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:02.007961 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bdbc7c-c717-4c63-b79a-672350b216fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:02.008661 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdbc7c-c717-4c63-b79a-672350b216fa None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:02.008941 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdbc7c-c717-4c63-b79a-672350b216fa None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:02.009211 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdbc7c-c717-4c63-b79a-672350b216fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:02.009937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bdbc7c-c717-4c63-b79a-672350b216fa None admin] RBAC: Adding query filter params (group.id=e943de99a6e9465cb060d45671dc02e6, role.id=ff2ed192587843e5b10b5a35949e4c5f, scope.project.id=59ae2524fa884ab7a9682a93d368b500) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:02.010245 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bdbc7c-c717-4c63-b79a-672350b216fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:02.013068 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bdbc7c-c717-4c63-b79a-672350b216fa None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:02.020093 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 190/372] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Sep 10 23:42:01 2022] GET /identity/v3/role_assignments?group.id=e943de99a6e9465cb060d45671dc02e6&scope.project.id=59ae2524fa884ab7a9682a93d368b500&role.id=ff2ed192587843e5b10b5a35949e4c5f => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:02.021386 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bdbc7c-c717-4c63-b79a-672350b216fa None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:02.182125 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b35d1-b754-4f8c-9c5d-644243ad095e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:02.233572 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:02.243277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bdbc7c-c717-4c63-b79a-672350b216fa None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:02.252488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca51c9-e7d2-4186-b458-8da7bbf980f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:02.252488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca51c9-e7d2-4186-b458-8da7bbf980f9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:02.252488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca51c9-e7d2-4186-b458-8da7bbf980f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:02.252488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca51c9-e7d2-4186-b458-8da7bbf980f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:02.252799 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca51c9-e7d2-4186-b458-8da7bbf980f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:02.252799 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca51c9-e7d2-4186-b458-8da7bbf980f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:02.252799 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca51c9-e7d2-4186-b458-8da7bbf980f9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:02.262096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 183/373] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:42:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:02.268011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca51c9-e7d2-4186-b458-8da7bbf980f9 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:02.279831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:02.295248 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63379c9-ab1b-4c1b-bb77-dbe5db89d0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:02.295895 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63379c9-ab1b-4c1b-bb77-dbe5db89d0ed None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:02.296172 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63379c9-ab1b-4c1b-bb77-dbe5db89d0ed None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:02.296438 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63379c9-ab1b-4c1b-bb77-dbe5db89d0ed None admin] PATH_INFO: `/v3/projects/15d53acb4e8c43348673ba66109e0371` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:02.299309 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63379c9-ab1b-4c1b-bb77-dbe5db89d0ed None admin] RBAC: Authorizing `identity:get_project(project_id=15d53acb4e8c43348673ba66109e0371)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:02.300741 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63379c9-ab1b-4c1b-bb77-dbe5db89d0ed None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:02.308591 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 191/374] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:42:02 2022] GET /identity/v3/projects/15d53acb4e8c43348673ba66109e0371 => generated 302 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:02.314559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63379c9-ab1b-4c1b-bb77-dbe5db89d0ed None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:02.433263 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63379c9-ab1b-4c1b-bb77-dbe5db89d0ed None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:02.464720 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca51c9-e7d2-4186-b458-8da7bbf980f9 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:02.474932 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4635018-d4a7-472d-9771-d762ce0a60a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:02.475563 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4635018-d4a7-472d-9771-d762ce0a60a3 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:02.475837 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4635018-d4a7-472d-9771-d762ce0a60a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:02.476106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4635018-d4a7-472d-9771-d762ce0a60a3 None admin] PATH_INFO: `/v3/groups/e943de99a6e9465cb060d45671dc02e6` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:02.477232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4635018-d4a7-472d-9771-d762ce0a60a3 None admin] RBAC: Authorizing `identity:get_group(group_id=e943de99a6e9465cb060d45671dc02e6)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:02.478846 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4635018-d4a7-472d-9771-d762ce0a60a3 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:02.480690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 192/375] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Sep 10 23:42:02 2022] GET /identity/v3/groups/e943de99a6e9465cb060d45671dc02e6 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:42:02.486723 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4635018-d4a7-472d-9771-d762ce0a60a3 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:02.563581 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7006561-c8de-4e4c-a756-6ad5644bc879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:02.564206 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7006561-c8de-4e4c-a756-6ad5644bc879 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:02.564491 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7006561-c8de-4e4c-a756-6ad5644bc879 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:02.564764 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7006561-c8de-4e4c-a756-6ad5644bc879 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:02.565154 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:02.597430 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4635018-d4a7-472d-9771-d762ce0a60a3 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:02.637870 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c52195-2177-47ac-b864-9768fd5bb4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:02.638525 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c52195-2177-47ac-b864-9768fd5bb4d6 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:02.639027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c52195-2177-47ac-b864-9768fd5bb4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:02.640199 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c52195-2177-47ac-b864-9768fd5bb4d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:02.640943 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c52195-2177-47ac-b864-9768fd5bb4d6 None admin] RBAC: Adding query filter params (group.id=e943de99a6e9465cb060d45671dc02e6, role.id=ff2ed192587843e5b10b5a35949e4c5f, scope.project.id=15d53acb4e8c43348673ba66109e0371) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:02.641252 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c52195-2177-47ac-b864-9768fd5bb4d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:02.642706 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c52195-2177-47ac-b864-9768fd5bb4d6 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:02.657113 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 193/376] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Sep 10 23:42:02 2022] GET /identity/v3/role_assignments?group.id=e943de99a6e9465cb060d45671dc02e6&scope.project.id=15d53acb4e8c43348673ba66109e0371&role.id=ff2ed192587843e5b10b5a35949e4c5f => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:42:02.686804 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:02.697453 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7006561-c8de-4e4c-a756-6ad5644bc879 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:02.698956 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7006561-c8de-4e4c-a756-6ad5644bc879 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:02.849664 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 184/377] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:42:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:42:02.855802 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c52195-2177-47ac-b864-9768fd5bb4d6 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:02.973826 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c52195-2177-47ac-b864-9768fd5bb4d6 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:03.000317 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:03.012972 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cea7d-65af-45a8-a2ee-a81b46f78af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:03.013758 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cea7d-65af-45a8-a2ee-a81b46f78af8 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:03.014027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cea7d-65af-45a8-a2ee-a81b46f78af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:03.014289 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cea7d-65af-45a8-a2ee-a81b46f78af8 None admin] PATH_INFO: `/v3/projects/ab88f917c063407982ac832bef946248` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:03.015489 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cea7d-65af-45a8-a2ee-a81b46f78af8 None admin] RBAC: Authorizing `identity:get_project(project_id=ab88f917c063407982ac832bef946248)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:03.016942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cea7d-65af-45a8-a2ee-a81b46f78af8 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:03.018994 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 194/378] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:42:02 2022] GET /identity/v3/projects/ab88f917c063407982ac832bef946248 => generated 344 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:03.024529 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7006561-c8de-4e4c-a756-6ad5644bc879 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:03.124353 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:03.133173 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7006561-c8de-4e4c-a756-6ad5644bc879 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:03.138848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3c396-0f14-4c0b-9c21-0aad03bbc3dc None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:03.145608 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3c396-0f14-4c0b-9c21-0aad03bbc3dc None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:03.145608 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3c396-0f14-4c0b-9c21-0aad03bbc3dc None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:03.145608 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 195/379] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:03.147307 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbc96a-1f50-4e6e-9c00-49db53b5de3b None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:03.147931 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbc96a-1f50-4e6e-9c00-49db53b5de3b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:03.148430 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbc96a-1f50-4e6e-9c00-49db53b5de3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:03.176516 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-cfdbc96a-1f50-4e6e-9c00-49db53b5de3b None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:03.178169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cfdbc96a-1f50-4e6e-9c00-49db53b5de3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:03.191678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad9ee2-59cd-4e20-88e8-3cc1d0b9cda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:03.192680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad9ee2-59cd-4e20-88e8-3cc1d0b9cda7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:03.193224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad9ee2-59cd-4e20-88e8-3cc1d0b9cda7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:03.193684 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad9ee2-59cd-4e20-88e8-3cc1d0b9cda7 None admin] PATH_INFO: `/v3/groups/ef6a74e02e9849ee853dbac9f29feedf` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:03.202573 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad9ee2-59cd-4e20-88e8-3cc1d0b9cda7 None admin] RBAC: Authorizing `identity:get_group(group_id=ef6a74e02e9849ee853dbac9f29feedf)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:03.204420 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad9ee2-59cd-4e20-88e8-3cc1d0b9cda7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:03.207353 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 185/380] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Sep 10 23:42:03 2022] GET /identity/v3/groups/ef6a74e02e9849ee853dbac9f29feedf => generated 229 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:03.213947 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad9ee2-59cd-4e20-88e8-3cc1d0b9cda7 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:03.322475 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 196/381] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:03.380576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad9ee2-59cd-4e20-88e8-3cc1d0b9cda7 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:03.429756 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b4441-86e0-4d43-92b7-13c12c3f74a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:03.430368 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b4441-86e0-4d43-92b7-13c12c3f74a6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:03.430650 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b4441-86e0-4d43-92b7-13c12c3f74a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:03.431024 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b4441-86e0-4d43-92b7-13c12c3f74a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:03.432006 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b4441-86e0-4d43-92b7-13c12c3f74a6 None admin] RBAC: Adding query filter params (group.id=ef6a74e02e9849ee853dbac9f29feedf, role.id=9dce48a2b53c482d931dcd064b07744e, scope.project.id=ab88f917c063407982ac832bef946248) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:03.432326 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b4441-86e0-4d43-92b7-13c12c3f74a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:03.433757 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b4441-86e0-4d43-92b7-13c12c3f74a6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:03.440410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 186/382] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Sep 10 23:42:03 2022] GET /identity/v3/role_assignments?group.id=ef6a74e02e9849ee853dbac9f29feedf&scope.project.id=ab88f917c063407982ac832bef946248&role.id=9dce48a2b53c482d931dcd064b07744e => generated 260 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:03.854905 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbc96a-1f50-4e6e-9c00-49db53b5de3b None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:03.978380 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbc96a-1f50-4e6e-9c00-49db53b5de3b None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:04.003396 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:04.018470 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b660b9-814d-4842-8d95-bda79275ca90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:04.019932 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b660b9-814d-4842-8d95-bda79275ca90 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:04.019932 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b660b9-814d-4842-8d95-bda79275ca90 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:04.019932 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b660b9-814d-4842-8d95-bda79275ca90 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:04.030902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b660b9-814d-4842-8d95-bda79275ca90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:04.035829 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-a7b660b9-814d-4842-8d95-bda79275ca90 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:42:04.035829 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:04.035829 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:04.035829 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:04.035829 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:04.035829 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:04.035829 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:04.035829 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:04.035829 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:04.035829 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:04.035829 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:04.035829 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:04.035829 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:04.035829 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:04.035829 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:04.035829 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:04.036812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:04.036812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:04.036812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:04.036812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:04.036812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:04.036812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:04.036812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:04.036812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:04.036812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:04.036812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:04.036812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:04.036812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:04.036812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:04.036812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:04.036812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:04.036812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:04.038232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:04.038232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:04.038232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:04.038232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:04.038232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:04.038232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:04.038232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:04.038232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:04.038232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:04.038232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:04.038232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:42:04.038232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:04.038232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 197/383] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:42:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:04.042720 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b4441-86e0-4d43-92b7-13c12c3f74a6 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:04.065419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:04.167119 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b4441-86e0-4d43-92b7-13c12c3f74a6 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:04.213397 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34910926-a120-45cd-8f9e-1b529a69a943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:04.214008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34910926-a120-45cd-8f9e-1b529a69a943 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:04.214286 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34910926-a120-45cd-8f9e-1b529a69a943 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:04.214557 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34910926-a120-45cd-8f9e-1b529a69a943 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:04.215255 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34910926-a120-45cd-8f9e-1b529a69a943 None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:04.215622 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34910926-a120-45cd-8f9e-1b529a69a943 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:04.217049 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34910926-a120-45cd-8f9e-1b529a69a943 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:04.231544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 187/384] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:42:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:04.232255 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b660b9-814d-4842-8d95-bda79275ca90 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:04.422244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b660b9-814d-4842-8d95-bda79275ca90 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:04.476291 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6581847-772b-49f7-bb35-22a2df42b61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:04.476935 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6581847-772b-49f7-bb35-22a2df42b61e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:04.477217 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6581847-772b-49f7-bb35-22a2df42b61e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:04.477492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6581847-772b-49f7-bb35-22a2df42b61e None admin] PATH_INFO: `/v3/groups/e943de99a6e9465cb060d45671dc02e6` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:04.478635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6581847-772b-49f7-bb35-22a2df42b61e None admin] RBAC: Authorizing `identity:get_group(group_id=e943de99a6e9465cb060d45671dc02e6)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:04.480120 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6581847-772b-49f7-bb35-22a2df42b61e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:04.482261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 198/385] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Sep 10 23:42:04 2022] GET /identity/v3/groups/e943de99a6e9465cb060d45671dc02e6 => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:04.487936 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34910926-a120-45cd-8f9e-1b529a69a943 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:04.515069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1da4e4-0445-40e0-a0fe-50ca29427f30 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:04.515392 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1da4e4-0445-40e0-a0fe-50ca29427f30 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:04.515600 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1da4e4-0445-40e0-a0fe-50ca29427f30 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:04.517194 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 199/386] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:42:04.521984 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9de85-c1f4-4829-8921-a26e5caa1db5 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:04.522262 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9de85-c1f4-4829-8921-a26e5caa1db5 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:04.522536 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9de85-c1f4-4829-8921-a26e5caa1db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:04.555041 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-83e9de85-c1f4-4829-8921-a26e5caa1db5 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:04.556243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-83e9de85-c1f4-4829-8921-a26e5caa1db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:04.647070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:04.671362 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:04.686748 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34910926-a120-45cd-8f9e-1b529a69a943 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:04.704511 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 200/387] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:04.756437 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4a44f-ce8d-4ad1-9a83-727a0e4c5746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:04.757195 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4a44f-ce8d-4ad1-9a83-727a0e4c5746 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:04.757482 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4a44f-ce8d-4ad1-9a83-727a0e4c5746 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:04.757761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4a44f-ce8d-4ad1-9a83-727a0e4c5746 None admin] PATH_INFO: `/v3/projects/59ae2524fa884ab7a9682a93d368b500` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:04.759465 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4a44f-ce8d-4ad1-9a83-727a0e4c5746 None admin] RBAC: Authorizing `identity:get_project(project_id=59ae2524fa884ab7a9682a93d368b500)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:04.764171 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4a44f-ce8d-4ad1-9a83-727a0e4c5746 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:04.770935 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 188/388] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:42:04 2022] GET /identity/v3/projects/59ae2524fa884ab7a9682a93d368b500 => generated 298 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:04.779585 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9de85-c1f4-4829-8921-a26e5caa1db5 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:04.942735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9de85-c1f4-4829-8921-a26e5caa1db5 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:04.999153 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3848da5d-df19-4fae-b8a2-9f8df32abf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:05.000641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848da5d-df19-4fae-b8a2-9f8df32abf54 None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:05.001129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848da5d-df19-4fae-b8a2-9f8df32abf54 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:05.001595 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848da5d-df19-4fae-b8a2-9f8df32abf54 None admin] PATH_INFO: `/v3/projects/59ae2524fa884ab7a9682a93d368b500/groups/e943de99a6e9465cb060d45671dc02e6/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:05.007453 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3848da5d-df19-4fae-b8a2-9f8df32abf54 None admin] RBAC: Authorizing `identity:create_grant(project_id=59ae2524fa884ab7a9682a93d368b500, group_id=e943de99a6e9465cb060d45671dc02e6, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:05.009330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3848da5d-df19-4fae-b8a2-9f8df32abf54 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:05.090511 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 201/389] 10.222.0.17 () {58 vars in 1442 bytes} [Sat Sep 10 23:42:04 2022] PUT /identity/v3/projects/59ae2524fa884ab7a9682a93d368b500/groups/e943de99a6e9465cb060d45671dc02e6/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:42:05.097708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93750b-186c-4740-98c7-f87dccbacc9b None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:05.098740 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93750b-186c-4740-98c7-f87dccbacc9b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:05.099221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93750b-186c-4740-98c7-f87dccbacc9b None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:05.100907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 189/390] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:05.114184 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb80dbb-ee7e-4829-8fbf-90fe397efd97 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:05.114879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb80dbb-ee7e-4829-8fbf-90fe397efd97 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:05.115367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb80dbb-ee7e-4829-8fbf-90fe397efd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:05.144106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-abb80dbb-ee7e-4829-8fbf-90fe397efd97 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:05.145577 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-abb80dbb-ee7e-4829-8fbf-90fe397efd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:05.152919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93750b-186c-4740-98c7-f87dccbacc9b None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:05.188122 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:05.272477 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 202/391] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:05.348386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93750b-186c-4740-98c7-f87dccbacc9b None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:05.399873 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ed5a1-0ed3-4b8e-9e2f-618509c28797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:05.400952 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ed5a1-0ed3-4b8e-9e2f-618509c28797 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:05.401437 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ed5a1-0ed3-4b8e-9e2f-618509c28797 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:05.401926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ed5a1-0ed3-4b8e-9e2f-618509c28797 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:05.414993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ed5a1-0ed3-4b8e-9e2f-618509c28797 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:05.421392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-956ed5a1-0ed3-4b8e-9e2f-618509c28797 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:42:05.421392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:05.421392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:05.421392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:05.421392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:05.421392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:05.421392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:05.421392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:05.421392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:05.421392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:05.421392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:05.421392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:05.421392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:05.421392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:05.421392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:05.421392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:05.422265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:05.422265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:05.422265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:05.422265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:05.422265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:05.422265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:05.422265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:05.422265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:05.422265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:05.422265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:05.422265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:05.422265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:05.422265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:05.422265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:05.422265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:05.422265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:05.423376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:05.423376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:05.423376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:05.423376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:05.423376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:05.423376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:05.423376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:05.423376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:05.423376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:05.423376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:05.423376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:42:05.423376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:05.423376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 190/392] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:42:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:05.429713 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb80dbb-ee7e-4829-8fbf-90fe397efd97 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:05.556059 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb80dbb-ee7e-4829-8fbf-90fe397efd97 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:05.595503 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ed5a1-0ed3-4b8e-9e2f-618509c28797 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:05.604375 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45503e1-2fae-4a2e-b3ae-4a8121a81328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:05.605390 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45503e1-2fae-4a2e-b3ae-4a8121a81328 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:05.605867 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45503e1-2fae-4a2e-b3ae-4a8121a81328 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:05.607116 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45503e1-2fae-4a2e-b3ae-4a8121a81328 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:05.608051 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45503e1-2fae-4a2e-b3ae-4a8121a81328 None admin] RBAC: Adding query filter params (name=member) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:05.608553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45503e1-2fae-4a2e-b3ae-4a8121a81328 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:05.610168 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45503e1-2fae-4a2e-b3ae-4a8121a81328 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:05.614158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:05.621291 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 203/393] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:42:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:05.627869 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45503e1-2fae-4a2e-b3ae-4a8121a81328 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:05.664191 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:05.767817 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45503e1-2fae-4a2e-b3ae-4a8121a81328 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:05.771473 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ed5a1-0ed3-4b8e-9e2f-618509c28797 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:05.805405 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b41ba0-4d0e-40b8-88c1-52e64b35a910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:05.806032 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b41ba0-4d0e-40b8-88c1-52e64b35a910 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:05.806362 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b41ba0-4d0e-40b8-88c1-52e64b35a910 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:05.806575 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b41ba0-4d0e-40b8-88c1-52e64b35a910 None admin] PATH_INFO: `/v3/groups/e943de99a6e9465cb060d45671dc02e6` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:05.808083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b41ba0-4d0e-40b8-88c1-52e64b35a910 None admin] RBAC: Authorizing `identity:get_group(group_id=e943de99a6e9465cb060d45671dc02e6)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:05.809910 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b41ba0-4d0e-40b8-88c1-52e64b35a910 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:05.814761 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 204/394] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Sep 10 23:42:05 2022] GET /identity/v3/groups/e943de99a6e9465cb060d45671dc02e6 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:05.816133 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc7161-35c6-4e3c-8120-823b89ac3e15 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:05.816419 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc7161-35c6-4e3c-8120-823b89ac3e15 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:05.816694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc7161-35c6-4e3c-8120-823b89ac3e15 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:05.818802 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 205/395] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:05.819469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc7161-35c6-4e3c-8120-823b89ac3e15 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:05.820021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c39dc-6eae-46fb-8ada-86f34bde00cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:05.820662 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c39dc-6eae-46fb-8ada-86f34bde00cc None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:05.820943 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c39dc-6eae-46fb-8ada-86f34bde00cc None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:05.821217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c39dc-6eae-46fb-8ada-86f34bde00cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:05.834443 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c39dc-6eae-46fb-8ada-86f34bde00cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:05.839328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-248c39dc-6eae-46fb-8ada-86f34bde00cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 23:42:05.839328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:05.839328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:05.839328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:05.839328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:05.839328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:05.839328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:05.839328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:05.839328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:05.839328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:05.839328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:05.839328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:05.839328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:05.839328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:05.839328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:05.839328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:05.840201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:05.840201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:05.840201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:05.840201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:05.840201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:05.840201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:05.840201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:05.840201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:05.840201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:05.840201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:05.840201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:05.840201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:05.840201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:05.840201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:05.840201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:05.840201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:05.841242 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:05.841242 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:05.841242 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:05.841242 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:05.841242 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:05.841242 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:05.841242 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:05.841242 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:05.841242 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:05.841242 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:05.841242 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 23:42:05.841242 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:05.841242 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 191/396] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Sep 10 23:42:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:05.842389 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4ef02-0a8d-4f36-a588-13c0b61f80a5 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:05.842389 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4ef02-0a8d-4f36-a588-13c0b61f80a5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:05.842699 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4ef02-0a8d-4f36-a588-13c0b61f80a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:05.861630 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-9fa4ef02-0a8d-4f36-a588-13c0b61f80a5 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:05.862848 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9fa4ef02-0a8d-4f36-a588-13c0b61f80a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:05.931232 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:05.936679 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 192/397] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:05.937593 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4ef02-0a8d-4f36-a588-13c0b61f80a5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:05.938628 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:05.943738 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc7161-35c6-4e3c-8120-823b89ac3e15 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:05.984659 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4fb19b-da0b-45ba-804e-672bd5fa7ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:05.985707 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4fb19b-da0b-45ba-804e-672bd5fa7ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:05.986162 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4fb19b-da0b-45ba-804e-672bd5fa7ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:05.986638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4fb19b-da0b-45ba-804e-672bd5fa7ca1 None admin] PATH_INFO: `/v3/projects/15d53acb4e8c43348673ba66109e0371` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:05.988045 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4fb19b-da0b-45ba-804e-672bd5fa7ca1 None admin] RBAC: Authorizing `identity:get_project(project_id=15d53acb4e8c43348673ba66109e0371)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:05.989927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4fb19b-da0b-45ba-804e-672bd5fa7ca1 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:05.992436 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 206/398] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:42:05 2022] GET /identity/v3/projects/15d53acb4e8c43348673ba66109e0371 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:05.999449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4fb19b-da0b-45ba-804e-672bd5fa7ca1 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:06.070286 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4ef02-0a8d-4f36-a588-13c0b61f80a5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:06.112904 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4fb19b-da0b-45ba-804e-672bd5fa7ca1 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:06.117388 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5f4b0-e55b-459c-b4ce-a32c605435c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:06.118520 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5f4b0-e55b-459c-b4ce-a32c605435c4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:06.118973 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5f4b0-e55b-459c-b4ce-a32c605435c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:06.119404 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5f4b0-e55b-459c-b4ce-a32c605435c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:06.120213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5f4b0-e55b-459c-b4ce-a32c605435c4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:06.120670 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5f4b0-e55b-459c-b4ce-a32c605435c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:06.123435 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5f4b0-e55b-459c-b4ce-a32c605435c4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:06.135381 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 193/399] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:06.142700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5f4b0-e55b-459c-b4ce-a32c605435c4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:06.164753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fffa49-02d8-4824-a0c0-fc2dbfb78b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:06.165471 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fffa49-02d8-4824-a0c0-fc2dbfb78b16 None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:06.165747 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fffa49-02d8-4824-a0c0-fc2dbfb78b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:06.166064 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fffa49-02d8-4824-a0c0-fc2dbfb78b16 None admin] PATH_INFO: `/v3/projects/15d53acb4e8c43348673ba66109e0371/groups/e943de99a6e9465cb060d45671dc02e6/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:06.170591 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fffa49-02d8-4824-a0c0-fc2dbfb78b16 None admin] RBAC: Authorizing `identity:create_grant(project_id=15d53acb4e8c43348673ba66109e0371, group_id=e943de99a6e9465cb060d45671dc02e6, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:06.172056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fffa49-02d8-4824-a0c0-fc2dbfb78b16 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:06.231433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:06.260582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 207/400] 10.222.0.17 () {58 vars in 1442 bytes} [Sat Sep 10 23:42:05 2022] PUT /identity/v3/projects/15d53acb4e8c43348673ba66109e0371/groups/e943de99a6e9465cb060d45671dc02e6/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 23:42:06.297261 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5f4b0-e55b-459c-b4ce-a32c605435c4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:06.312845 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fffa49-02d8-4824-a0c0-fc2dbfb78b16 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:06.351126 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:06.386417 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b4421-15b3-4203-9fa9-244212fa2eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:06.387162 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b4421-15b3-4203-9fa9-244212fa2eba None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:06.388224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b4421-15b3-4203-9fa9-244212fa2eba None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:06.388623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b4421-15b3-4203-9fa9-244212fa2eba None admin] PATH_INFO: `/v3/groups/e943de99a6e9465cb060d45671dc02e6` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:06.389995 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b4421-15b3-4203-9fa9-244212fa2eba None admin] RBAC: Authorizing `identity:get_group(group_id=e943de99a6e9465cb060d45671dc02e6)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:06.393564 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b4421-15b3-4203-9fa9-244212fa2eba None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:06.395627 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 194/401] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Sep 10 23:42:06 2022] GET /identity/v3/groups/e943de99a6e9465cb060d45671dc02e6 => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:42:06.396854 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69bb64-1b57-44c5-bffa-a117ca9ae78b None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:06.397133 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69bb64-1b57-44c5-bffa-a117ca9ae78b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:06.397405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69bb64-1b57-44c5-bffa-a117ca9ae78b None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:06.398800 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 195/402] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:06.404111 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69bb64-1b57-44c5-bffa-a117ca9ae78b None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:06.458401 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fffa49-02d8-4824-a0c0-fc2dbfb78b16 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:06.513099 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e10c71-e5f3-4a3b-b515-91c1a057e90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:06.514535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e10c71-e5f3-4a3b-b515-91c1a057e90a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:06.515026 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e10c71-e5f3-4a3b-b515-91c1a057e90a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:06.515878 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e10c71-e5f3-4a3b-b515-91c1a057e90a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:06.530630 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e10c71-e5f3-4a3b-b515-91c1a057e90a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:06.538022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-16e10c71-e5f3-4a3b-b515-91c1a057e90a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 23:42:06.538022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:06.538022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:06.538022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:06.538022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:06.538022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:06.538022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:06.538022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:06.538022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:06.538022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:06.538022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:06.538022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:06.538022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:06.538022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:06.538022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:06.538022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:06.538930 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:06.538930 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:06.538930 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:06.538930 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:06.538930 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:06.538930 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:06.538930 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:06.538930 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:06.538930 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:06.538930 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:06.538930 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:06.538930 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:06.538930 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:06.538930 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:06.538930 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:06.538930 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:06.539734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:06.539734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:06.539734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:06.539734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:06.539734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:06.539734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:06.539734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:06.539734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:06.539734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:06.539734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:06.539734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 23:42:06.539734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:06.541489 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69bb64-1b57-44c5-bffa-a117ca9ae78b None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:06.542190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 208/403] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Sep 10 23:42:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:42:06.544629 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57c8f5-f73f-459b-a5f0-0f5be0f19568 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:06.546037 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57c8f5-f73f-459b-a5f0-0f5be0f19568 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:06.546464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57c8f5-f73f-459b-a5f0-0f5be0f19568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:06.570321 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-ba57c8f5-f73f-459b-a5f0-0f5be0f19568 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:06.572668 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ba57c8f5-f73f-459b-a5f0-0f5be0f19568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:06.574078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:06.585832 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e53a3-243b-4b2f-8904-01a6ea298f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:06.587182 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e53a3-243b-4b2f-8904-01a6ea298f88 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:06.587628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e53a3-243b-4b2f-8904-01a6ea298f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:06.588464 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e53a3-243b-4b2f-8904-01a6ea298f88 None admin] PATH_INFO: `/v3/projects/59ae2524fa884ab7a9682a93d368b500` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:06.590094 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e53a3-243b-4b2f-8904-01a6ea298f88 None admin] RBAC: Authorizing `identity:get_project(project_id=59ae2524fa884ab7a9682a93d368b500)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:06.592204 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e53a3-243b-4b2f-8904-01a6ea298f88 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:06.595005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 196/404] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:42:06 2022] GET /identity/v3/projects/59ae2524fa884ab7a9682a93d368b500 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:06.596134 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e53a3-243b-4b2f-8904-01a6ea298f88 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:06.651510 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 209/405] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:06.652723 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57c8f5-f73f-459b-a5f0-0f5be0f19568 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:06.689169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:06.766237 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e53a3-243b-4b2f-8904-01a6ea298f88 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:06.820058 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57c8f5-f73f-459b-a5f0-0f5be0f19568 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:06.841527 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4c94f-0759-4b61-88a9-59236fa9a928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:06.842149 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4c94f-0759-4b61-88a9-59236fa9a928 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:06.842423 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4c94f-0759-4b61-88a9-59236fa9a928 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:06.842750 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4c94f-0759-4b61-88a9-59236fa9a928 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:06.843433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b4c94f-0759-4b61-88a9-59236fa9a928 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:06.843732 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b4c94f-0759-4b61-88a9-59236fa9a928 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:06.845364 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b4c94f-0759-4b61-88a9-59236fa9a928 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:06.865685 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 197/406] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:06.869806 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4c94f-0759-4b61-88a9-59236fa9a928 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:06.888410 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07d2ba-43fe-4d26-a235-b8fb831eb4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:06.889357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07d2ba-43fe-4d26-a235-b8fb831eb4af None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:06.889634 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07d2ba-43fe-4d26-a235-b8fb831eb4af None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:06.892141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07d2ba-43fe-4d26-a235-b8fb831eb4af None admin] PATH_INFO: `/v3/projects/59ae2524fa884ab7a9682a93d368b500/groups/e943de99a6e9465cb060d45671dc02e6/roles/ff2ed192587843e5b10b5a35949e4c5f` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:06.896162 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc07d2ba-43fe-4d26-a235-b8fb831eb4af None admin] RBAC: Authorizing `identity:create_grant(project_id=59ae2524fa884ab7a9682a93d368b500, group_id=e943de99a6e9465cb060d45671dc02e6, role_id=ff2ed192587843e5b10b5a35949e4c5f)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:06.900791 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc07d2ba-43fe-4d26-a235-b8fb831eb4af None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:07.005512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 210/407] 10.222.0.17 () {58 vars in 1442 bytes} [Sat Sep 10 23:42:06 2022] PUT /identity/v3/projects/59ae2524fa884ab7a9682a93d368b500/groups/e943de99a6e9465cb060d45671dc02e6/roles/ff2ed192587843e5b10b5a35949e4c5f => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:42:07.009842 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07d2ba-43fe-4d26-a235-b8fb831eb4af None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:07.027528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:07.132778 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4c94f-0759-4b61-88a9-59236fa9a928 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:07.254596 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0485e-e36f-4a80-a0b7-11872abccb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:07.258742 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0485e-e36f-4a80-a0b7-11872abccb88 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:07.258742 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0485e-e36f-4a80-a0b7-11872abccb88 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:07.258742 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0485e-e36f-4a80-a0b7-11872abccb88 None admin] PATH_INFO: `/v3/groups/e943de99a6e9465cb060d45671dc02e6` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:07.258742 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0485e-e36f-4a80-a0b7-11872abccb88 None admin] RBAC: Authorizing `identity:get_group(group_id=e943de99a6e9465cb060d45671dc02e6)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:07.259049 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0485e-e36f-4a80-a0b7-11872abccb88 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:07.260408 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 198/408] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Sep 10 23:42:06 2022] GET /identity/v3/groups/e943de99a6e9465cb060d45671dc02e6 => generated 226 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:07.268390 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:07.269032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0485e-e36f-4a80-a0b7-11872abccb88 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:07.323824 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07d2ba-43fe-4d26-a235-b8fb831eb4af None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:07.444565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f155d-b4ad-43ef-90dd-5eca6d83f036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:07.445753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f155d-b4ad-43ef-90dd-5eca6d83f036 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:07.445753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f155d-b4ad-43ef-90dd-5eca6d83f036 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:07.445753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f155d-b4ad-43ef-90dd-5eca6d83f036 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:07.482422 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f155d-b4ad-43ef-90dd-5eca6d83f036 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:07.490102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-452f155d-b4ad-43ef-90dd-5eca6d83f036 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:42:07.490102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:07.490102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:07.490102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:07.490102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:07.490102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:07.490102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:07.490102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:07.490102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:07.490102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:07.490102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:07.490102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:07.490102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:07.490102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:07.490102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:07.490102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:07.491074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:07.491074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:07.491074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:07.491074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:07.491074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:07.491074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:07.491074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:07.491074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:07.491074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:07.491074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:07.491074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:07.491074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:07.491074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:07.491074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:07.491074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:07.491074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:07.491855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:07.491855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:07.491855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:07.491855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:07.491855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:07.491855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:07.491855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:07.491855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:07.491855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:07.491855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:07.491855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:42:07.491855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:07.508924 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 211/409] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:42:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 503 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:07.521829 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f155d-b4ad-43ef-90dd-5eca6d83f036 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:07.587741 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0485e-e36f-4a80-a0b7-11872abccb88 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:07.617848 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:07.634747 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98691718-84e7-4bca-857f-6aacb971dd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:07.635382 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98691718-84e7-4bca-857f-6aacb971dd8b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:07.644353 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98691718-84e7-4bca-857f-6aacb971dd8b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:07.644353 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98691718-84e7-4bca-857f-6aacb971dd8b None admin] PATH_INFO: `/v3/projects/15d53acb4e8c43348673ba66109e0371` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:07.644353 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98691718-84e7-4bca-857f-6aacb971dd8b None admin] RBAC: Authorizing `identity:get_project(project_id=15d53acb4e8c43348673ba66109e0371)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:07.644353 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98691718-84e7-4bca-857f-6aacb971dd8b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:07.644353 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 199/410] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:42:07 2022] GET /identity/v3/projects/15d53acb4e8c43348673ba66109e0371 => generated 302 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:07.650691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98691718-84e7-4bca-857f-6aacb971dd8b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:07.755476 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98691718-84e7-4bca-857f-6aacb971dd8b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:07.772637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f155d-b4ad-43ef-90dd-5eca6d83f036 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:07.797113 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f627ce-b8bc-4ac2-8185-ed5156fd9bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:07.797770 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f627ce-b8bc-4ac2-8185-ed5156fd9bcc None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:07.798044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f627ce-b8bc-4ac2-8185-ed5156fd9bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:07.798311 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f627ce-b8bc-4ac2-8185-ed5156fd9bcc None admin] PATH_INFO: `/v3/projects/15d53acb4e8c43348673ba66109e0371/groups/e943de99a6e9465cb060d45671dc02e6/roles/ff2ed192587843e5b10b5a35949e4c5f` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:07.802201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f627ce-b8bc-4ac2-8185-ed5156fd9bcc None admin] RBAC: Authorizing `identity:create_grant(project_id=15d53acb4e8c43348673ba66109e0371, group_id=e943de99a6e9465cb060d45671dc02e6, role_id=ff2ed192587843e5b10b5a35949e4c5f)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:07.803683 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f627ce-b8bc-4ac2-8185-ed5156fd9bcc None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:07.829079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca980d5-cbc3-48aa-b956-08019a7559e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:07.829816 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca980d5-cbc3-48aa-b956-08019a7559e7 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:07.830028 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca980d5-cbc3-48aa-b956-08019a7559e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:07.830346 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca980d5-cbc3-48aa-b956-08019a7559e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:07.831139 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca980d5-cbc3-48aa-b956-08019a7559e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:07.831446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca980d5-cbc3-48aa-b956-08019a7559e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:07.832992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca980d5-cbc3-48aa-b956-08019a7559e7 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:07.843646 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 212/411] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:42:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:07.844196 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:07.850986 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca980d5-cbc3-48aa-b956-08019a7559e7 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:07.897382 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 200/412] 10.222.0.17 () {58 vars in 1442 bytes} [Sat Sep 10 23:42:07 2022] PUT /identity/v3/projects/15d53acb4e8c43348673ba66109e0371/groups/e943de99a6e9465cb060d45671dc02e6/roles/ff2ed192587843e5b10b5a35949e4c5f => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:42:07.979125 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca980d5-cbc3-48aa-b956-08019a7559e7 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:08.016549 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e20e6a-7f82-4715-b120-b49824cedbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:08.017216 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e20e6a-7f82-4715-b120-b49824cedbb4 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:08.017464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e20e6a-7f82-4715-b120-b49824cedbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:08.017736 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e20e6a-7f82-4715-b120-b49824cedbb4 None admin] PATH_INFO: `/v3/groups/ef6a74e02e9849ee853dbac9f29feedf` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:08.019045 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e20e6a-7f82-4715-b120-b49824cedbb4 None admin] RBAC: Authorizing `identity:get_group(group_id=ef6a74e02e9849ee853dbac9f29feedf)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:08.020486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e20e6a-7f82-4715-b120-b49824cedbb4 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:08.022474 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 213/413] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Sep 10 23:42:07 2022] GET /identity/v3/groups/ef6a74e02e9849ee853dbac9f29feedf => generated 229 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:42:08.028963 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f627ce-b8bc-4ac2-8185-ed5156fd9bcc None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:08.041515 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e0780-50fa-4b11-9f5e-3fb60c88df34 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:08.041843 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e0780-50fa-4b11-9f5e-3fb60c88df34 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:08.042071 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e0780-50fa-4b11-9f5e-3fb60c88df34 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:08.043777 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 214/414] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:08.052156 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6867c3e-28a7-4c1c-beb2-34a1104b32c3 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:08.052480 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6867c3e-28a7-4c1c-beb2-34a1104b32c3 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:08.052822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6867c3e-28a7-4c1c-beb2-34a1104b32c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:08.083604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b6867c3e-28a7-4c1c-beb2-34a1104b32c3 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:08.084933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b6867c3e-28a7-4c1c-beb2-34a1104b32c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:08.102495 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:08.166923 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:08.182862 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 215/415] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:08.187615 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f627ce-b8bc-4ac2-8185-ed5156fd9bcc None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:08.242936 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f520f0c-0ee2-4317-a3c1-212b041536ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:08.243809 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f520f0c-0ee2-4317-a3c1-212b041536ba None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:08.244222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f520f0c-0ee2-4317-a3c1-212b041536ba None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:08.244627 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f520f0c-0ee2-4317-a3c1-212b041536ba None admin] PATH_INFO: `/v3/projects/ab88f917c063407982ac832bef946248` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:08.245919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f520f0c-0ee2-4317-a3c1-212b041536ba None admin] RBAC: Authorizing `identity:get_project(project_id=ab88f917c063407982ac832bef946248)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:08.248064 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f520f0c-0ee2-4317-a3c1-212b041536ba None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:08.250172 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 201/416] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:42:08 2022] GET /identity/v3/projects/ab88f917c063407982ac832bef946248 => generated 344 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:08.259814 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6867c3e-28a7-4c1c-beb2-34a1104b32c3 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:08.415522 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6867c3e-28a7-4c1c-beb2-34a1104b32c3 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:08.455595 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec6604-d99e-462d-bb08-9c5ffa182705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:08.456254 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec6604-d99e-462d-bb08-9c5ffa182705 None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:08.456526 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec6604-d99e-462d-bb08-9c5ffa182705 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:08.456791 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec6604-d99e-462d-bb08-9c5ffa182705 None admin] PATH_INFO: `/v3/projects/ab88f917c063407982ac832bef946248/groups/ef6a74e02e9849ee853dbac9f29feedf/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:08.463984 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec6604-d99e-462d-bb08-9c5ffa182705 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab88f917c063407982ac832bef946248, group_id=ef6a74e02e9849ee853dbac9f29feedf, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:08.466055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec6604-d99e-462d-bb08-9c5ffa182705 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:08.471522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f520f0c-0ee2-4317-a3c1-212b041536ba None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:08.549698 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 216/417] 10.222.0.17 () {58 vars in 1442 bytes} [Sat Sep 10 23:42:08 2022] PUT /identity/v3/projects/ab88f917c063407982ac832bef946248/groups/ef6a74e02e9849ee853dbac9f29feedf/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:42:08.605246 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f520f0c-0ee2-4317-a3c1-212b041536ba None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:08.632771 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:08.652146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6c3f3-363a-4ce7-8db2-cc7d58688dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:08.653881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6c3f3-363a-4ce7-8db2-cc7d58688dbf None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:08.654363 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6c3f3-363a-4ce7-8db2-cc7d58688dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:08.654852 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6c3f3-363a-4ce7-8db2-cc7d58688dbf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:08.665998 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef6c3f3-363a-4ce7-8db2-cc7d58688dbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:08.675493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-7ef6c3f3-363a-4ce7-8db2-cc7d58688dbf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:42:08.675493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:08.675493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:08.675493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:08.675493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:08.675493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:08.675493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:08.675493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:08.675493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:08.675493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:08.675493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:08.675493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:08.675493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:08.675493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:08.675493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:08.675493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:08.676419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:08.676419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:08.676419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:08.676419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:08.676419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:08.676419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:08.676419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:08.676419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:08.676419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:08.676419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:08.676419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:08.676419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:08.676419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:08.676419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:08.676419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:08.676419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:08.677224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:08.677224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:08.677224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:08.677224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:08.677224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:08.677224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:08.677224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:08.677224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:08.677224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:08.677224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:08.677224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:42:08.677224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:08.679597 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 202/418] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:42:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:42:08.686299 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec6604-d99e-462d-bb08-9c5ffa182705 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:08.821903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:08.913948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec6604-d99e-462d-bb08-9c5ffa182705 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:08.947716 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f03f83-f138-40ed-a25b-a96fe6c59d6a None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:08.948337 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f03f83-f138-40ed-a25b-a96fe6c59d6a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:08.948837 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f03f83-f138-40ed-a25b-a96fe6c59d6a None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:08.950965 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 203/419] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:08.957264 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6645b-4d76-47f8-9818-c0e140289a2a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:08.962027 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6645b-4d76-47f8-9818-c0e140289a2a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:08.962027 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6645b-4d76-47f8-9818-c0e140289a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:08.980863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf0d10-cd15-4657-8acf-052006ffc08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:08.981593 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf0d10-cd15-4657-8acf-052006ffc08f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:08.981871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf0d10-cd15-4657-8acf-052006ffc08f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:08.982134 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf0d10-cd15-4657-8acf-052006ffc08f None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:08.982832 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf0d10-cd15-4657-8acf-052006ffc08f None admin] RBAC: Adding query filter params (name=member) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:08.983134 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf0d10-cd15-4657-8acf-052006ffc08f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:08.984504 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf0d10-cd15-4657-8acf-052006ffc08f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:08.994989 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d4b6645b-4d76-47f8-9818-c0e140289a2a None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:08.996208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d4b6645b-4d76-47f8-9818-c0e140289a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:09.001818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 217/420] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:42:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:09.011677 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf0d10-cd15-4657-8acf-052006ffc08f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:09.094975 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 204/421] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:09.161911 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf0d10-cd15-4657-8acf-052006ffc08f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:09.179710 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:09.197792 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc056f5-4727-467c-be37-8396e107014f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:09.198392 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc056f5-4727-467c-be37-8396e107014f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:09.198685 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc056f5-4727-467c-be37-8396e107014f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:09.198954 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc056f5-4727-467c-be37-8396e107014f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:09.295015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc056f5-4727-467c-be37-8396e107014f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:09.296492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc056f5-4727-467c-be37-8396e107014f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:09.431161 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 218/422] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:42:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:42:09.437867 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6645b-4d76-47f8-9818-c0e140289a2a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:09.534530 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc056f5-4727-467c-be37-8396e107014f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:09.567880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:09.589857 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6645b-4d76-47f8-9818-c0e140289a2a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:09.610913 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:09.648794 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e58f46-fee0-4965-872f-f2d46a99caa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:09.649431 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e58f46-fee0-4965-872f-f2d46a99caa5 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:09.649707 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e58f46-fee0-4965-872f-f2d46a99caa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:09.649973 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e58f46-fee0-4965-872f-f2d46a99caa5 None admin] PATH_INFO: `/v3/projects/59ae2524fa884ab7a9682a93d368b500` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:09.651190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e58f46-fee0-4965-872f-f2d46a99caa5 None admin] RBAC: Authorizing `identity:get_project(project_id=59ae2524fa884ab7a9682a93d368b500)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:09.652647 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e58f46-fee0-4965-872f-f2d46a99caa5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:09.655251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 205/423] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:42:09 2022] GET /identity/v3/projects/59ae2524fa884ab7a9682a93d368b500 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:09.657871 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5602ba-ad24-4d27-9c1d-07db59b473cb None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:09.658317 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5602ba-ad24-4d27-9c1d-07db59b473cb None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:09.658824 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5602ba-ad24-4d27-9c1d-07db59b473cb None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:09.660213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 206/424] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:42:09.664155 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5602ba-ad24-4d27-9c1d-07db59b473cb None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:09.704981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc056f5-4727-467c-be37-8396e107014f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:09.748097 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccd43d-6769-4142-b45f-394c8596b21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:09.749098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccd43d-6769-4142-b45f-394c8596b21a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:09.749575 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccd43d-6769-4142-b45f-394c8596b21a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:09.750106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccd43d-6769-4142-b45f-394c8596b21a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:09.761185 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebccd43d-6769-4142-b45f-394c8596b21a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:09.766773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-ebccd43d-6769-4142-b45f-394c8596b21a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:42:09.766773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:09.766773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:09.766773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:09.766773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:09.766773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:09.766773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:09.766773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:09.766773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:09.766773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:09.766773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:09.766773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:09.766773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:09.766773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:09.766773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:09.766773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:09.767737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:09.767737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:09.767737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:09.767737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:09.767737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:09.767737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:09.767737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:09.767737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:09.767737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:09.767737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:09.767737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:09.767737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:09.767737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:09.767737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:09.767737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:09.767737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:09.768537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:09.768537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:09.768537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:09.768537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:09.768537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:09.768537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:09.768537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:09.768537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:09.768537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:09.768537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:09.768537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 23:42:09.768537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:09.771174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 219/425] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:42:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:09.772607 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a717c-a639-4933-924f-1a5ec1f66c30 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:09.773101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a717c-a639-4933-924f-1a5ec1f66c30 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:09.773565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a717c-a639-4933-924f-1a5ec1f66c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:09.790280 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5602ba-ad24-4d27-9c1d-07db59b473cb None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:09.793037 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-689a717c-a639-4933-924f-1a5ec1f66c30 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:09.794799 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-689a717c-a639-4933-924f-1a5ec1f66c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:09.835813 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f973ae1-0a7d-4cd0-b4e6-fac494d4d9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:09.836439 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f973ae1-0a7d-4cd0-b4e6-fac494d4d9e2 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:09.836713 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f973ae1-0a7d-4cd0-b4e6-fac494d4d9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:09.836979 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f973ae1-0a7d-4cd0-b4e6-fac494d4d9e2 None admin] PATH_INFO: `/v3/groups/e943de99a6e9465cb060d45671dc02e6` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:09.838101 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f973ae1-0a7d-4cd0-b4e6-fac494d4d9e2 None admin] RBAC: Authorizing `identity:get_group(group_id=e943de99a6e9465cb060d45671dc02e6)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:09.839565 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f973ae1-0a7d-4cd0-b4e6-fac494d4d9e2 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:09.846733 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 207/426] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Sep 10 23:42:09 2022] GET /identity/v3/groups/e943de99a6e9465cb060d45671dc02e6 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:09.847872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f973ae1-0a7d-4cd0-b4e6-fac494d4d9e2 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:09.882795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 220/427] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:09.883845 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a717c-a639-4933-924f-1a5ec1f66c30 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:09.908470 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:09.909568 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:10.003410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f973ae1-0a7d-4cd0-b4e6-fac494d4d9e2 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:10.065153 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ee7dc-b1b9-4bd0-a30b-f3f6e5d04fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:10.065710 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ee7dc-b1b9-4bd0-a30b-f3f6e5d04fce None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:10.065992 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ee7dc-b1b9-4bd0-a30b-f3f6e5d04fce None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:10.066261 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ee7dc-b1b9-4bd0-a30b-f3f6e5d04fce None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:10.066944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ee7dc-b1b9-4bd0-a30b-f3f6e5d04fce None admin] RBAC: Adding query filter params (name=member) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:10.067246 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ee7dc-b1b9-4bd0-a30b-f3f6e5d04fce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:10.068630 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ee7dc-b1b9-4bd0-a30b-f3f6e5d04fce None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:10.082654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a717c-a639-4933-924f-1a5ec1f66c30 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:10.095000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 208/428] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:42:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:10.095925 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ee7dc-b1b9-4bd0-a30b-f3f6e5d04fce None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:10.118498 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a1167-172e-4811-9d7f-d5484abce3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:10.119291 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a1167-172e-4811-9d7f-d5484abce3d9 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:10.119656 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a1167-172e-4811-9d7f-d5484abce3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:10.120037 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a1167-172e-4811-9d7f-d5484abce3d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:10.120827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a1167-172e-4811-9d7f-d5484abce3d9 None admin] RBAC: Adding query filter params (group.id=e943de99a6e9465cb060d45671dc02e6, role.id=824f5fe1f02b447382190a8c236c27db, scope.project.id=59ae2524fa884ab7a9682a93d368b500) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:10.121187 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a1167-172e-4811-9d7f-d5484abce3d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:10.122977 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a1167-172e-4811-9d7f-d5484abce3d9 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:10.129443 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 221/429] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Sep 10 23:42:09 2022] GET /identity/v3/role_assignments?group.id=e943de99a6e9465cb060d45671dc02e6&scope.project.id=59ae2524fa884ab7a9682a93d368b500&role.id=824f5fe1f02b447382190a8c236c27db => generated 611 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:10.250800 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ee7dc-b1b9-4bd0-a30b-f3f6e5d04fce None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:10.265500 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:10.294122 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c2cb8-8fa9-4fe2-befa-8669b1893733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:10.294772 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c2cb8-8fa9-4fe2-befa-8669b1893733 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:10.295137 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c2cb8-8fa9-4fe2-befa-8669b1893733 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:10.295441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c2cb8-8fa9-4fe2-befa-8669b1893733 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:10.323739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a1167-172e-4811-9d7f-d5484abce3d9 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:10.399976 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c2cb8-8fa9-4fe2-befa-8669b1893733 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:10.401777 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c2cb8-8fa9-4fe2-befa-8669b1893733 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:10.436866 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:10.478796 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a1167-172e-4811-9d7f-d5484abce3d9 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:10.519550 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474da7f-1bbc-4157-af8e-119da332109b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:10.520923 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474da7f-1bbc-4157-af8e-119da332109b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:10.521364 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474da7f-1bbc-4157-af8e-119da332109b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:10.521795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474da7f-1bbc-4157-af8e-119da332109b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:10.523810 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 209/430] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:42:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:42:10.530772 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c2cb8-8fa9-4fe2-befa-8669b1893733 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:10.537152 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6474da7f-1bbc-4157-af8e-119da332109b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:10.542722 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-6474da7f-1bbc-4157-af8e-119da332109b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 23:42:10.542722 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:10.542722 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:10.542722 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:10.542722 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:10.542722 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:10.542722 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:10.542722 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:10.542722 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:10.542722 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:10.542722 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:10.542722 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:10.542722 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:10.542722 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:10.542722 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:10.542722 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:10.543757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:10.543757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:10.543757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:10.543757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:10.543757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:10.543757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:10.543757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:10.543757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:10.543757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:10.543757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:10.543757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:10.543757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:10.543757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:10.543757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:10.543757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:10.543757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:10.544589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:10.544589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:10.544589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:10.544589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:10.544589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:10.544589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:10.544589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:10.544589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:10.544589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:10.544589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:10.544589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 23:42:10.544589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:10.547154 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 222/431] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Sep 10 23:42:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:42:10.548525 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:10.553704 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474da7f-1bbc-4157-af8e-119da332109b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:10.657464 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c2cb8-8fa9-4fe2-befa-8669b1893733 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:10.677192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474da7f-1bbc-4157-af8e-119da332109b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:10.701895 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430161e5-db59-4e85-a13f-08cad98c5ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:10.702703 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430161e5-db59-4e85-a13f-08cad98c5ffd None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:10.703089 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430161e5-db59-4e85-a13f-08cad98c5ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:10.703460 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430161e5-db59-4e85-a13f-08cad98c5ffd None admin] PATH_INFO: `/v3/projects/15d53acb4e8c43348673ba66109e0371` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:10.705134 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430161e5-db59-4e85-a13f-08cad98c5ffd None admin] RBAC: Authorizing `identity:get_project(project_id=15d53acb4e8c43348673ba66109e0371)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:10.706595 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430161e5-db59-4e85-a13f-08cad98c5ffd None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:10.708522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 210/432] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:42:10 2022] GET /identity/v3/projects/15d53acb4e8c43348673ba66109e0371 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:10.709599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f156410-4151-45a2-9e3b-bff0df0d6549 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:10.709881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f156410-4151-45a2-9e3b-bff0df0d6549 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:10.710151 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f156410-4151-45a2-9e3b-bff0df0d6549 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:10.711559 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 211/433] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:10.714283 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:10.716780 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f156410-4151-45a2-9e3b-bff0df0d6549 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:10.723990 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc80a8-5fa3-43b1-ab0c-cda2e712346d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:10.724601 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc80a8-5fa3-43b1-ab0c-cda2e712346d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:10.724871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc80a8-5fa3-43b1-ab0c-cda2e712346d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:10.725139 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc80a8-5fa3-43b1-ab0c-cda2e712346d None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:10.725797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc80a8-5fa3-43b1-ab0c-cda2e712346d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:10.726089 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc80a8-5fa3-43b1-ab0c-cda2e712346d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:10.727618 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc80a8-5fa3-43b1-ab0c-cda2e712346d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:10.737183 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 223/434] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:10.738610 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b72c92-37f1-43a4-99aa-480a8dc5165d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:10.738931 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b72c92-37f1-43a4-99aa-480a8dc5165d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:10.739201 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b72c92-37f1-43a4-99aa-480a8dc5165d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:10.759734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-41b72c92-37f1-43a4-99aa-480a8dc5165d None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:10.761249 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-41b72c92-37f1-43a4-99aa-480a8dc5165d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:10.829259 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:10.850057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f156410-4151-45a2-9e3b-bff0df0d6549 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:10.857441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 224/435] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:10.866744 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b72c92-37f1-43a4-99aa-480a8dc5165d None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:10.894025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d300d53-eec4-497d-9046-c921c7694619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:10.894833 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d300d53-eec4-497d-9046-c921c7694619 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:10.895170 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d300d53-eec4-497d-9046-c921c7694619 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:10.895502 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d300d53-eec4-497d-9046-c921c7694619 None admin] PATH_INFO: `/v3/groups/e943de99a6e9465cb060d45671dc02e6` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:10.896684 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d300d53-eec4-497d-9046-c921c7694619 None admin] RBAC: Authorizing `identity:get_group(group_id=e943de99a6e9465cb060d45671dc02e6)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:10.898177 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d300d53-eec4-497d-9046-c921c7694619 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:10.905222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 212/436] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Sep 10 23:42:10 2022] GET /identity/v3/groups/e943de99a6e9465cb060d45671dc02e6 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:10.906697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d300d53-eec4-497d-9046-c921c7694619 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:10.992487 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b72c92-37f1-43a4-99aa-480a8dc5165d None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:11.023079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:11.038806 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d1c8e-c05a-43e6-aebd-906442a03d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:11.039463 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d1c8e-c05a-43e6-aebd-906442a03d2f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:11.041613 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d1c8e-c05a-43e6-aebd-906442a03d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:11.041613 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d1c8e-c05a-43e6-aebd-906442a03d2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:11.046303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d300d53-eec4-497d-9046-c921c7694619 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:11.082674 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a30256f-d128-4fc1-a892-4c23430c2304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:11.083298 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30256f-d128-4fc1-a892-4c23430c2304 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:11.083643 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30256f-d128-4fc1-a892-4c23430c2304 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:11.084053 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30256f-d128-4fc1-a892-4c23430c2304 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:11.084921 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a30256f-d128-4fc1-a892-4c23430c2304 None admin] RBAC: Adding query filter params (group.id=e943de99a6e9465cb060d45671dc02e6, role.id=824f5fe1f02b447382190a8c236c27db, scope.project.id=15d53acb4e8c43348673ba66109e0371) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:11.085228 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a30256f-d128-4fc1-a892-4c23430c2304 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:11.086777 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a30256f-d128-4fc1-a892-4c23430c2304 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:11.098821 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 213/437] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Sep 10 23:42:10 2022] GET /identity/v3/role_assignments?group.id=e943de99a6e9465cb060d45671dc02e6&scope.project.id=15d53acb4e8c43348673ba66109e0371&role.id=824f5fe1f02b447382190a8c236c27db => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:42:11.125162 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d1c8e-c05a-43e6-aebd-906442a03d2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:11.127541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d1c8e-c05a-43e6-aebd-906442a03d2f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:11.167316 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a30256f-d128-4fc1-a892-4c23430c2304 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:11.184858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:11.257907 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 225/438] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:42:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:42:11.263602 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d1c8e-c05a-43e6-aebd-906442a03d2f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:11.308600 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:11.315576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a30256f-d128-4fc1-a892-4c23430c2304 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:11.359935 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c2007-6cb3-427d-aa8a-45215acb06ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:11.360478 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c2007-6cb3-427d-aa8a-45215acb06ff None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:11.360752 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c2007-6cb3-427d-aa8a-45215acb06ff None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:11.361018 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c2007-6cb3-427d-aa8a-45215acb06ff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:11.373527 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c2007-6cb3-427d-aa8a-45215acb06ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:11.378458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-263c2007-6cb3-427d-aa8a-45215acb06ff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 23:42:11.378458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:11.378458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:11.378458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:11.378458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:11.378458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:11.378458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:11.378458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:11.378458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:11.378458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:11.378458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:11.378458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:11.378458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:11.378458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:11.378458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:11.378458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:11.379414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:11.379414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:11.379414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:11.379414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:11.379414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:11.379414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:11.379414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:11.379414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:11.379414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:11.379414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:11.379414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:11.379414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:11.379414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:11.379414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:11.379414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:11.379414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:11.380218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:11.380218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:11.380218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:11.380218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:11.380218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:11.380218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:11.380218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:11.380218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:11.380218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:11.380218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:11.380218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 23:42:11.380218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:11.380218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 214/439] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Sep 10 23:42:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:42:11.381048 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70b7ff-adb8-4d88-9914-1195c426e0d9 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:11.381282 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70b7ff-adb8-4d88-9914-1195c426e0d9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:11.381561 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70b7ff-adb8-4d88-9914-1195c426e0d9 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:11.382824 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 215/440] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:11.385261 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70b7ff-adb8-4d88-9914-1195c426e0d9 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:11.392368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d1c8e-c05a-43e6-aebd-906442a03d2f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:11.425842 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625dbce0-06af-4001-8f1e-f4f792dc5061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:11.426459 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625dbce0-06af-4001-8f1e-f4f792dc5061 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:11.426760 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625dbce0-06af-4001-8f1e-f4f792dc5061 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:11.427026 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625dbce0-06af-4001-8f1e-f4f792dc5061 None admin] PATH_INFO: `/v3/projects/59ae2524fa884ab7a9682a93d368b500` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:11.428132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625dbce0-06af-4001-8f1e-f4f792dc5061 None admin] RBAC: Authorizing `identity:get_project(project_id=59ae2524fa884ab7a9682a93d368b500)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:11.429573 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625dbce0-06af-4001-8f1e-f4f792dc5061 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:11.431290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 226/441] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:42:11 2022] GET /identity/v3/projects/59ae2524fa884ab7a9682a93d368b500 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:42:11.433282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607b376-861d-4932-b5fb-a39602d66204 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:11.433560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607b376-861d-4932-b5fb-a39602d66204 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:11.434193 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607b376-861d-4932-b5fb-a39602d66204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:11.451710 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-4607b376-861d-4932-b5fb-a39602d66204 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:11.452876 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4607b376-861d-4932-b5fb-a39602d66204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:11.456347 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:11.488167 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70b7ff-adb8-4d88-9914-1195c426e0d9 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:11.519303 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 227/442] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:11.520165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4607b376-861d-4932-b5fb-a39602d66204 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:11.529171 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b187ffa-3c85-41c8-84ac-25dc6aaa730d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:11.529776 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b187ffa-3c85-41c8-84ac-25dc6aaa730d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:11.530048 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b187ffa-3c85-41c8-84ac-25dc6aaa730d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:11.530314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b187ffa-3c85-41c8-84ac-25dc6aaa730d None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:11.532554 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b187ffa-3c85-41c8-84ac-25dc6aaa730d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:11.532554 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b187ffa-3c85-41c8-84ac-25dc6aaa730d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:11.532755 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b187ffa-3c85-41c8-84ac-25dc6aaa730d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:11.540862 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 216/443] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:42:11.547112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b187ffa-3c85-41c8-84ac-25dc6aaa730d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:11.560637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:11.637005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4607b376-861d-4932-b5fb-a39602d66204 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:11.652428 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b187ffa-3c85-41c8-84ac-25dc6aaa730d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:11.672534 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825bff8-596a-437f-80ff-6844643f92bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:11.673163 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825bff8-596a-437f-80ff-6844643f92bc None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:11.673491 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825bff8-596a-437f-80ff-6844643f92bc None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:11.673883 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825bff8-596a-437f-80ff-6844643f92bc None admin] PATH_INFO: `/v3/groups/e943de99a6e9465cb060d45671dc02e6` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:11.675011 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3825bff8-596a-437f-80ff-6844643f92bc None admin] RBAC: Authorizing `identity:get_group(group_id=e943de99a6e9465cb060d45671dc02e6)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:11.676541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3825bff8-596a-437f-80ff-6844643f92bc None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:11.683136 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 228/444] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Sep 10 23:42:11 2022] GET /identity/v3/groups/e943de99a6e9465cb060d45671dc02e6 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:11.684258 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825bff8-596a-437f-80ff-6844643f92bc None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:11.692543 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3627a8f-8825-4a75-bae4-20ee432d04a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:11.693141 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3627a8f-8825-4a75-bae4-20ee432d04a9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:11.693872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3627a8f-8825-4a75-bae4-20ee432d04a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:11.694146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3627a8f-8825-4a75-bae4-20ee432d04a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:11.697373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:11.770612 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3627a8f-8825-4a75-bae4-20ee432d04a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:11.772406 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3627a8f-8825-4a75-bae4-20ee432d04a9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:11.784820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:11.795676 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825bff8-596a-437f-80ff-6844643f92bc None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:11.830557 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d02ae3b-0e0c-48a7-be4c-200e86068250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:11.831546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02ae3b-0e0c-48a7-be4c-200e86068250 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:11.832027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02ae3b-0e0c-48a7-be4c-200e86068250 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:11.832500 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02ae3b-0e0c-48a7-be4c-200e86068250 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:11.833764 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d02ae3b-0e0c-48a7-be4c-200e86068250 None admin] RBAC: Adding query filter params (group.id=e943de99a6e9465cb060d45671dc02e6, role.id=ff2ed192587843e5b10b5a35949e4c5f, scope.project.id=59ae2524fa884ab7a9682a93d368b500) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:11.834322 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d02ae3b-0e0c-48a7-be4c-200e86068250 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:11.836838 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d02ae3b-0e0c-48a7-be4c-200e86068250 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:11.844281 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 229/445] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Sep 10 23:42:11 2022] GET /identity/v3/role_assignments?group.id=e943de99a6e9465cb060d45671dc02e6&scope.project.id=59ae2524fa884ab7a9682a93d368b500&role.id=ff2ed192587843e5b10b5a35949e4c5f => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:11.845062 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d02ae3b-0e0c-48a7-be4c-200e86068250 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:11.883152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 217/446] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:42:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 10 23:42:11.884028 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3627a8f-8825-4a75-bae4-20ee432d04a9 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:11.923633 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:11.970589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d02ae3b-0e0c-48a7-be4c-200e86068250 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:12.000032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3627a8f-8825-4a75-bae4-20ee432d04a9 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:12.014177 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de719bd-7052-47eb-a839-2c847609fc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:12.014853 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de719bd-7052-47eb-a839-2c847609fc91 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:12.015197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de719bd-7052-47eb-a839-2c847609fc91 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:12.015466 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de719bd-7052-47eb-a839-2c847609fc91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:12.027378 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de719bd-7052-47eb-a839-2c847609fc91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:12.032709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-5de719bd-7052-47eb-a839-2c847609fc91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:42:12.032709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:12.032709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:12.032709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:12.032709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:12.032709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:12.032709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:12.032709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:12.032709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:12.032709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:12.032709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:12.032709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:12.032709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:12.032709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:12.032709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:12.032709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:12.033596 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:12.033596 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:12.033596 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:12.033596 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:12.033596 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:12.033596 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:12.033596 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:12.033596 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:12.033596 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:12.033596 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:12.033596 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:12.033596 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:12.033596 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:12.033596 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:12.033596 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:12.033596 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:12.034501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:12.034501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:12.034501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:12.034501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:12.034501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:12.034501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:12.034501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:12.034501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:12.034501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:12.034501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:12.034501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:42:12.034501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:12.034501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 230/447] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:42:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:12.039645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de719bd-7052-47eb-a839-2c847609fc91 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:12.047232 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221ba38-7239-4719-ae9a-cdd00eb98a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:12.047422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221ba38-7239-4719-ae9a-cdd00eb98a4e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:12.047675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221ba38-7239-4719-ae9a-cdd00eb98a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:12.047947 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221ba38-7239-4719-ae9a-cdd00eb98a4e None admin] PATH_INFO: `/v3/projects/15d53acb4e8c43348673ba66109e0371` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:12.049073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b221ba38-7239-4719-ae9a-cdd00eb98a4e None admin] RBAC: Authorizing `identity:get_project(project_id=15d53acb4e8c43348673ba66109e0371)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:12.050522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b221ba38-7239-4719-ae9a-cdd00eb98a4e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:12.052369 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 218/448] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:42:11 2022] GET /identity/v3/projects/15d53acb4e8c43348673ba66109e0371 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:12.057844 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221ba38-7239-4719-ae9a-cdd00eb98a4e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:12.058197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:12.150870 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de719bd-7052-47eb-a839-2c847609fc91 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:12.181156 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221ba38-7239-4719-ae9a-cdd00eb98a4e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:12.183248 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:12.189501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e1fca-f93d-4142-8f81-d3dd7903cf74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:12.190324 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e1fca-f93d-4142-8f81-d3dd7903cf74 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:12.191102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e1fca-f93d-4142-8f81-d3dd7903cf74 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:12.191374 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e1fca-f93d-4142-8f81-d3dd7903cf74 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:12.196427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e1fca-f93d-4142-8f81-d3dd7903cf74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:12.196427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e1fca-f93d-4142-8f81-d3dd7903cf74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:12.196427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e1fca-f93d-4142-8f81-d3dd7903cf74 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:12.204640 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 231/449] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:42:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:42:12.209684 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e1fca-f93d-4142-8f81-d3dd7903cf74 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:12.224298 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f2eebf-80ae-49cc-99b0-05687c08a38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:12.224929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2eebf-80ae-49cc-99b0-05687c08a38d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:12.225204 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2eebf-80ae-49cc-99b0-05687c08a38d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:12.226245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2eebf-80ae-49cc-99b0-05687c08a38d None admin] PATH_INFO: `/v3/groups/e943de99a6e9465cb060d45671dc02e6` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:12.227695 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f2eebf-80ae-49cc-99b0-05687c08a38d None admin] RBAC: Authorizing `identity:get_group(group_id=e943de99a6e9465cb060d45671dc02e6)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:12.228841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f2eebf-80ae-49cc-99b0-05687c08a38d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:12.230554 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 219/450] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Sep 10 23:42:12 2022] GET /identity/v3/groups/e943de99a6e9465cb060d45671dc02e6 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:42:12.236382 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f2eebf-80ae-49cc-99b0-05687c08a38d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:12.301039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:12.326609 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e1fca-f93d-4142-8f81-d3dd7903cf74 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:12.363775 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f2eebf-80ae-49cc-99b0-05687c08a38d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:12.369459 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1ded4-ad55-404c-916e-98a3da535ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:12.370139 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1ded4-ad55-404c-916e-98a3da535ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:12.370480 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1ded4-ad55-404c-916e-98a3da535ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:12.370908 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1ded4-ad55-404c-916e-98a3da535ce5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:12.404926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e34a3-9d0d-47d4-951f-373ed5339748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:12.405541 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e34a3-9d0d-47d4-951f-373ed5339748 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:12.405825 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e34a3-9d0d-47d4-951f-373ed5339748 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:12.406099 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e34a3-9d0d-47d4-951f-373ed5339748 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:12.406901 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e34a3-9d0d-47d4-951f-373ed5339748 None admin] RBAC: Adding query filter params (group.id=e943de99a6e9465cb060d45671dc02e6, role.id=ff2ed192587843e5b10b5a35949e4c5f, scope.project.id=15d53acb4e8c43348673ba66109e0371) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:12.407480 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e34a3-9d0d-47d4-951f-373ed5339748 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:12.408646 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e34a3-9d0d-47d4-951f-373ed5339748 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:12.418296 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 220/451] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Sep 10 23:42:12 2022] GET /identity/v3/role_assignments?group.id=e943de99a6e9465cb060d45671dc02e6&scope.project.id=15d53acb4e8c43348673ba66109e0371&role.id=ff2ed192587843e5b10b5a35949e4c5f => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:42:12.458721 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1ded4-ad55-404c-916e-98a3da535ce5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:12.460118 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1ded4-ad55-404c-916e-98a3da535ce5 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:12.540294 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:12.566116 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 232/452] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:42:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 10 23:42:12.571897 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e34a3-9d0d-47d4-951f-373ed5339748 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:12.607710 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:12.676304 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e34a3-9d0d-47d4-951f-373ed5339748 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:12.710113 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902fc616-4bac-4944-ac85-94580982ef9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:12.710654 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902fc616-4bac-4944-ac85-94580982ef9c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:12.710951 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902fc616-4bac-4944-ac85-94580982ef9c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:12.711219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902fc616-4bac-4944-ac85-94580982ef9c None admin] PATH_INFO: `/v3/projects/ab88f917c063407982ac832bef946248` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:12.712314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902fc616-4bac-4944-ac85-94580982ef9c None admin] RBAC: Authorizing `identity:get_project(project_id=ab88f917c063407982ac832bef946248)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:12.713744 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902fc616-4bac-4944-ac85-94580982ef9c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:12.715416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 221/453] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:42:12 2022] GET /identity/v3/projects/ab88f917c063407982ac832bef946248 => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:12.721099 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1ded4-ad55-404c-916e-98a3da535ce5 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:12.819195 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1ded4-ad55-404c-916e-98a3da535ce5 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:12.851304 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9fc8b7-cb42-494b-905e-c8ae33677a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:12.851907 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9fc8b7-cb42-494b-905e-c8ae33677a7d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:12.852178 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9fc8b7-cb42-494b-905e-c8ae33677a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:12.852441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9fc8b7-cb42-494b-905e-c8ae33677a7d None admin] PATH_INFO: `/v3/groups/ef6a74e02e9849ee853dbac9f29feedf` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:12.853508 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9fc8b7-cb42-494b-905e-c8ae33677a7d None admin] RBAC: Authorizing `identity:get_group(group_id=ef6a74e02e9849ee853dbac9f29feedf)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:12.854963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9fc8b7-cb42-494b-905e-c8ae33677a7d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:12.856543 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 233/454] 10.222.0.17 () {56 vars in 1180 bytes} [Sat Sep 10 23:42:12 2022] GET /identity/v3/groups/ef6a74e02e9849ee853dbac9f29feedf => generated 229 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:12.862377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902fc616-4bac-4944-ac85-94580982ef9c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:12.960012 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902fc616-4bac-4944-ac85-94580982ef9c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:12.963956 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:12.992773 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737ebdd-b991-4924-ac5d-633c3d9c8701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:12.993358 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737ebdd-b991-4924-ac5d-633c3d9c8701 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:12.993637 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737ebdd-b991-4924-ac5d-633c3d9c8701 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:12.993908 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737ebdd-b991-4924-ac5d-633c3d9c8701 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:12.994627 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737ebdd-b991-4924-ac5d-633c3d9c8701 None admin] RBAC: Adding query filter params (group.id=ef6a74e02e9849ee853dbac9f29feedf, role.id=9dce48a2b53c482d931dcd064b07744e, scope.project.id=ab88f917c063407982ac832bef946248) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:12.994975 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737ebdd-b991-4924-ac5d-633c3d9c8701 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:12.997035 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737ebdd-b991-4924-ac5d-633c3d9c8701 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:13.002833 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 222/455] 10.222.0.17 () {56 vars in 1376 bytes} [Sat Sep 10 23:42:12 2022] GET /identity/v3/role_assignments?group.id=ef6a74e02e9849ee853dbac9f29feedf&scope.project.id=ab88f917c063407982ac832bef946248&role.id=9dce48a2b53c482d931dcd064b07744e => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:15.668881 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e78ad-00ab-427a-8842-55d7f5416304 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:15.669443 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e78ad-00ab-427a-8842-55d7f5416304 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:15.669443 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e78ad-00ab-427a-8842-55d7f5416304 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:15.671231 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 234/456] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:15.677105 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed872f0-40e5-4659-aa68-424360157cba None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:15.677606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed872f0-40e5-4659-aa68-424360157cba None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:15.677678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed872f0-40e5-4659-aa68-424360157cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:15.707954 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1ed872f0-40e5-4659-aa68-424360157cba None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:15.709183 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1ed872f0-40e5-4659-aa68-424360157cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:15.801845 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 223/457] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:15.886411 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8351623-48d6-4af2-bd8b-cd52f796f275 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:15.887155 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8351623-48d6-4af2-bd8b-cd52f796f275 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:15.887643 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8351623-48d6-4af2-bd8b-cd52f796f275 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:15.889494 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 235/458] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:15.900180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5c2ef-4808-4410-84db-097aa0f1e546 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:15.900463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5c2ef-4808-4410-84db-097aa0f1e546 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:15.900738 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5c2ef-4808-4410-84db-097aa0f1e546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:15.923729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-aaa5c2ef-4808-4410-84db-097aa0f1e546 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:15.924829 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-aaa5c2ef-4808-4410-84db-097aa0f1e546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:15.932617 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289b54e-333f-4d7d-8a01-aa88089a2ea1 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:15.932902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289b54e-333f-4d7d-8a01-aa88089a2ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:15.933179 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289b54e-333f-4d7d-8a01-aa88089a2ea1 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:15.934897 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 236/459] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:15.941381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9244b2-fae6-4fb2-9a3a-dfad8294e074 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:15.941660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9244b2-fae6-4fb2-9a3a-dfad8294e074 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:15.941934 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9244b2-fae6-4fb2-9a3a-dfad8294e074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:15.971354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-ba9244b2-fae6-4fb2-9a3a-dfad8294e074 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:15.972541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ba9244b2-fae6-4fb2-9a3a-dfad8294e074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:16.048802 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:16.049902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 224/460] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:16.089600 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 237/461] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:16.129778 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5c2ef-4808-4410-84db-097aa0f1e546 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:16.165898 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:16.261564 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5c2ef-4808-4410-84db-097aa0f1e546 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:16.309779 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0de4b-4c01-49a6-a63b-b3d6abd38d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:16.310684 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0de4b-4c01-49a6-a63b-b3d6abd38d96 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:16.311201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0de4b-4c01-49a6-a63b-b3d6abd38d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:16.311681 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0de4b-4c01-49a6-a63b-b3d6abd38d96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:16.325889 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b0de4b-4c01-49a6-a63b-b3d6abd38d96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:16.327646 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b0de4b-4c01-49a6-a63b-b3d6abd38d96 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:16.345642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-87b0de4b-4c01-49a6-a63b-b3d6abd38d96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:16.345642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:16.345642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:16.345642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:16.345642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:16.345642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:16.345642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:16.345642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:16.345642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:16.345642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:16.345642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:16.345642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:16.345642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:16.345642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:16.345642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:16.345642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:16.346576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:16.346576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:16.346576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:16.346576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:16.346576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:16.346576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:16.346576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:16.346576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:16.346576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:16.346576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:16.346576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:16.346576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:16.346576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:16.346576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:16.346576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:16.346576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:16.346576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:16.347445 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:16.347445 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:16.347445 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:16.349214 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 225/462] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:16.355217 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9244b2-fae6-4fb2-9a3a-dfad8294e074 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:16.444116 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0de4b-4c01-49a6-a63b-b3d6abd38d96 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:16.500288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9244b2-fae6-4fb2-9a3a-dfad8294e074 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:16.543788 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba7ceb-eda5-4aff-bfc2-470f31138eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:16.544393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba7ceb-eda5-4aff-bfc2-470f31138eee None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:16.544666 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba7ceb-eda5-4aff-bfc2-470f31138eee None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:16.544995 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba7ceb-eda5-4aff-bfc2-470f31138eee None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:16.545634 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba7ceb-eda5-4aff-bfc2-470f31138eee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:16.546406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba7ceb-eda5-4aff-bfc2-470f31138eee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:16.548271 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:16.548864 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba7ceb-eda5-4aff-bfc2-470f31138eee None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:16.564342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 238/463] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:16.565535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba7ceb-eda5-4aff-bfc2-470f31138eee None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:16.577419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0de4b-4c01-49a6-a63b-b3d6abd38d96 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:16.601267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:16.615972 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaacd1d-89c1-43d4-8435-686b217182f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:16.616596 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaacd1d-89c1-43d4-8435-686b217182f4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:16.616871 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaacd1d-89c1-43d4-8435-686b217182f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:16.617138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaacd1d-89c1-43d4-8435-686b217182f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:16.629184 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaacd1d-89c1-43d4-8435-686b217182f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:16.630870 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaacd1d-89c1-43d4-8435-686b217182f4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:16.645924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-efaacd1d-89c1-43d4-8435-686b217182f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:16.645924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:16.645924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:16.645924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:16.645924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:16.645924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:16.645924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:16.645924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:16.645924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:16.645924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:16.645924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:16.645924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:16.645924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:16.645924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:16.645924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:16.645924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:16.646879 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:16.646879 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:16.646879 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:16.646879 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:16.646879 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:16.646879 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:16.646879 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:16.646879 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:16.646879 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:16.646879 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:16.646879 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:16.646879 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:16.646879 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:16.646879 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:16.646879 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:16.646879 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:16.646879 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:16.647708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:16.647708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:16.647708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:16.652033 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 226/464] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:42:16.653673 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaacd1d-89c1-43d4-8435-686b217182f4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:16.674114 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba7ceb-eda5-4aff-bfc2-470f31138eee None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:16.712474 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e023f98-5050-4f3a-bdab-b79c4655b414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:16.713536 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e023f98-5050-4f3a-bdab-b79c4655b414 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:16.714031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e023f98-5050-4f3a-bdab-b79c4655b414 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:16.714490 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e023f98-5050-4f3a-bdab-b79c4655b414 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:16.728211 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e023f98-5050-4f3a-bdab-b79c4655b414 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:16.730058 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e023f98-5050-4f3a-bdab-b79c4655b414 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:16.745903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-5e023f98-5050-4f3a-bdab-b79c4655b414 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:16.745903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:16.745903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:16.745903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:16.745903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:16.745903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:16.745903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:16.745903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:16.745903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:16.745903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:16.745903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:16.745903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:16.745903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:16.745903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:16.745903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:16.745903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:16.747402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:16.747402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:16.747402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:16.747402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:16.747402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:16.747402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:16.747402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:16.747402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:16.747402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:16.747402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:16.747402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:16.747402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:16.747402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:16.747402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:16.747402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:16.747402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:16.747402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:16.748233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:16.748233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:16.748233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:16.750363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 239/465] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:16.751449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e023f98-5050-4f3a-bdab-b79c4655b414 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:16.775676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaacd1d-89c1-43d4-8435-686b217182f4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:16.813770 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:16.820962 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff454cc4-4ad2-4bdd-8c01-a8aa7f059e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:16.821614 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff454cc4-4ad2-4bdd-8c01-a8aa7f059e2d None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:16.823463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff454cc4-4ad2-4bdd-8c01-a8aa7f059e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:16.823463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff454cc4-4ad2-4bdd-8c01-a8aa7f059e2d None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:16.823463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff454cc4-4ad2-4bdd-8c01-a8aa7f059e2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:16.824460 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff454cc4-4ad2-4bdd-8c01-a8aa7f059e2d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:16.853521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:16.868461 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e023f98-5050-4f3a-bdab-b79c4655b414 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:16.900941 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99053696-ce55-428f-a057-cf4dab5233e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:16.901531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99053696-ce55-428f-a057-cf4dab5233e1 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:16.901801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99053696-ce55-428f-a057-cf4dab5233e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:16.902063 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99053696-ce55-428f-a057-cf4dab5233e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:16.903352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99053696-ce55-428f-a057-cf4dab5233e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:16.903675 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99053696-ce55-428f-a057-cf4dab5233e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:16.905100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99053696-ce55-428f-a057-cf4dab5233e1 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:16.916583 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 240/466] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:16.919204 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99053696-ce55-428f-a057-cf4dab5233e1 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:16.946054 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 227/467] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Sep 10 23:42:16 2022] POST /identity/v3/users => generated 246 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:42:16.946054 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1258e75-8d68-4d16-bc1e-52fc4ad15521 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:16.946054 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1258e75-8d68-4d16-bc1e-52fc4ad15521 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:16.946054 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1258e75-8d68-4d16-bc1e-52fc4ad15521 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:16.946054 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 228/468] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:16.946054 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1258e75-8d68-4d16-bc1e-52fc4ad15521 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:17.050040 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99053696-ce55-428f-a057-cf4dab5233e1 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:17.089170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abb5fd-8f19-4dec-af37-6c6cee8165ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:17.089801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abb5fd-8f19-4dec-af37-6c6cee8165ea None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:17.090778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abb5fd-8f19-4dec-af37-6c6cee8165ea None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:17.091144 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abb5fd-8f19-4dec-af37-6c6cee8165ea None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:17.091865 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abb5fd-8f19-4dec-af37-6c6cee8165ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:17.092245 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abb5fd-8f19-4dec-af37-6c6cee8165ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:17.093871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abb5fd-8f19-4dec-af37-6c6cee8165ea None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:17.111374 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 241/469] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:17.112512 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1258e75-8d68-4d16-bc1e-52fc4ad15521 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:17.113161 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8061b1-1dc2-4bba-b54b-516a3a5ed217 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:17.113513 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8061b1-1dc2-4bba-b54b-516a3a5ed217 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:17.113788 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8061b1-1dc2-4bba-b54b-516a3a5ed217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:17.126842 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:17.132734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-3b8061b1-1dc2-4bba-b54b-516a3a5ed217 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:17.133947 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3b8061b1-1dc2-4bba-b54b-516a3a5ed217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:17.154475 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da56b8b8-8b16-4d51-8294-d478c005ff17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:17.155804 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56b8b8-8b16-4d51-8294-d478c005ff17 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:17.155804 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56b8b8-8b16-4d51-8294-d478c005ff17 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:17.155804 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56b8b8-8b16-4d51-8294-d478c005ff17 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:17.162427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da56b8b8-8b16-4d51-8294-d478c005ff17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:17.162427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da56b8b8-8b16-4d51-8294-d478c005ff17 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:17.205524 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 242/470] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:17.207053 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edebaf-7185-4930-bf38-2e09f2e36eb1 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:17.207399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edebaf-7185-4930-bf38-2e09f2e36eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:17.207602 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edebaf-7185-4930-bf38-2e09f2e36eb1 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:17.208817 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 243/471] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:17.209550 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edebaf-7185-4930-bf38-2e09f2e36eb1 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:17.248863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 229/472] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Sep 10 23:42:16 2022] POST /identity/v3/users => generated 248 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:42:17.250285 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccb488-05a4-4428-9b47-08c7c32745c8 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:17.250568 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccb488-05a4-4428-9b47-08c7c32745c8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:17.250861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccb488-05a4-4428-9b47-08c7c32745c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:17.277219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c8ccb488-05a4-4428-9b47-08c7c32745c8 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:17.279671 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c8ccb488-05a4-4428-9b47-08c7c32745c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:17.338717 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edebaf-7185-4930-bf38-2e09f2e36eb1 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:17.382884 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273bd9a-0367-4d25-abdf-cb3999ef2403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:17.384182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273bd9a-0367-4d25-abdf-cb3999ef2403 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:17.384661 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273bd9a-0367-4d25-abdf-cb3999ef2403 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:17.385128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273bd9a-0367-4d25-abdf-cb3999ef2403 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:17.385991 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273bd9a-0367-4d25-abdf-cb3999ef2403 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:17.389105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273bd9a-0367-4d25-abdf-cb3999ef2403 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:17.399615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:17.405809 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:17.458186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 230/473] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:17.525059 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 244/474] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Sep 10 23:42:17 2022] POST /identity/v3/users => generated 248 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:42:17.554983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ccb488-05a4-4428-9b47-08c7c32745c8 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:17.686459 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ccb488-05a4-4428-9b47-08c7c32745c8 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:17.734188 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11109eba-773d-4bd1-907c-5efa7714f5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:17.735154 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11109eba-773d-4bd1-907c-5efa7714f5b7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:17.735573 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11109eba-773d-4bd1-907c-5efa7714f5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:17.736004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11109eba-773d-4bd1-907c-5efa7714f5b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:17.747262 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11109eba-773d-4bd1-907c-5efa7714f5b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:17.749213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11109eba-773d-4bd1-907c-5efa7714f5b7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:17.765030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-11109eba-773d-4bd1-907c-5efa7714f5b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:17.765030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:17.765030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:17.765030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:17.765030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:17.765030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:17.765030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:17.765030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:17.765030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:17.765030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:17.765030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:17.765030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:17.765030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:17.765030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:17.765030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:17.765030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:17.766159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:17.766159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:17.766159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:17.766159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:17.766159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:17.766159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:17.766159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:17.766159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:17.766159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:17.766159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:17.766159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:17.766159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:17.766159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:17.766159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:17.766159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:17.766159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:17.766159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:17.767192 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:17.767192 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:17.767192 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:17.768922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 231/475] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:17.770703 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273bd9a-0367-4d25-abdf-cb3999ef2403 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:17.776076 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11109eba-773d-4bd1-907c-5efa7714f5b7 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:17.812656 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:17.900511 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273bd9a-0367-4d25-abdf-cb3999ef2403 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:17.906318 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11109eba-773d-4bd1-907c-5efa7714f5b7 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:17.943558 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127dba6-dccc-4ef4-a985-f10a078e6e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:17.944484 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127dba6-dccc-4ef4-a985-f10a078e6e7c None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:17.944959 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127dba6-dccc-4ef4-a985-f10a078e6e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:17.945414 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127dba6-dccc-4ef4-a985-f10a078e6e7c None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:17.946268 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3127dba6-dccc-4ef4-a985-f10a078e6e7c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:17.948118 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3127dba6-dccc-4ef4-a985-f10a078e6e7c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:17.948699 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbfb3eb-89b1-4cc9-83ba-a1fb4fb6635c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:17.953243 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbfb3eb-89b1-4cc9-83ba-a1fb4fb6635c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:17.953243 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbfb3eb-89b1-4cc9-83ba-a1fb4fb6635c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:17.953243 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbfb3eb-89b1-4cc9-83ba-a1fb4fb6635c None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:17.953243 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbfb3eb-89b1-4cc9-83ba-a1fb4fb6635c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:17.953243 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbfb3eb-89b1-4cc9-83ba-a1fb4fb6635c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:17.953243 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbfb3eb-89b1-4cc9-83ba-a1fb4fb6635c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:17.967706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 232/476] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:17.976987 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbfb3eb-89b1-4cc9-83ba-a1fb4fb6635c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:17.994753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.sql.core [None req-3127dba6-dccc-4ef4-a985-f10a078e6e7c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 10 23:42:17.994753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 10 23:42:17.994753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [parameters: {'id': 'b978db94120844c5b2171342301d258a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 10 23:42:17.994753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81573) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 10 23:42:18.016579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-3127dba6-dccc-4ef4-a985-f10a078e6e7c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 10 23:42:18.016579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:18.016579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 23:42:18.016579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 23:42:18.016579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 23:42:18.016579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 23:42:18.016579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 23:42:18.016579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 23:42:18.016579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 23:42:18.016579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application conn.query(q) Sep 10 23:42:18.016579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 23:42:18.016579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 23:42:18.016579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 23:42:18.016579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application result.read() Sep 10 23:42:18.016579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 23:42:18.017541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 23:42:18.017541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 23:42:18.017541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 23:42:18.017541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 23:42:18.017541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 23:42:18.017541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 23:42:18.017541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 23:42:18.017541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 10 23:42:18.017541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:18.017541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 10 23:42:18.017541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:18.017541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:18.017541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 10 23:42:18.017541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 10 23:42:18.017541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 10 23:42:18.017541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return ref.to_dict() Sep 10 23:42:18.017541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 23:42:18.017541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application next(self.gen) Sep 10 23:42:18.018450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 10 23:42:18.018450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application yield resource Sep 10 23:42:18.018450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 23:42:18.018450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application next(self.gen) Sep 10 23:42:18.018450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 10 23:42:18.018450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self.session.rollback() Sep 10 23:42:18.018450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 10 23:42:18.018450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self.force_reraise() Sep 10 23:42:18.018450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 10 23:42:18.018450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise self.value Sep 10 23:42:18.018450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 10 23:42:18.018450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 10 23:42:18.018450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 10 23:42:18.018450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application session.commit() Sep 10 23:42:18.018450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 10 23:42:18.018450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 10 23:42:18.018450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 10 23:42:18.019391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._prepare_impl() Sep 10 23:42:18.019391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 10 23:42:18.019391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self.session.flush() Sep 10 23:42:18.019391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 10 23:42:18.019391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._flush(objects) Sep 10 23:42:18.019391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 10 23:42:18.019391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 10 23:42:18.019391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 10 23:42:18.019391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application compat.raise_( Sep 10 23:42:18.019391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 23:42:18.019391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception Sep 10 23:42:18.019391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 10 23:42:18.019391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application flush_context.execute() Sep 10 23:42:18.019391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 10 23:42:18.019391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rec.execute(self) Sep 10 23:42:18.019391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 10 23:42:18.019391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 10 23:42:18.020381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 10 23:42:18.020381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 10 23:42:18.020381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 10 23:42:18.020381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 10 23:42:18.020381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 10 23:42:18.020381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 10 23:42:18.020381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 10 23:42:18.020381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 10 23:42:18.020381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 10 23:42:18.020381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 10 23:42:18.020381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 10 23:42:18.020381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 10 23:42:18.020381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 10 23:42:18.020381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 10 23:42:18.020381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 23:42:18.020381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception Sep 10 23:42:18.021202 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 23:42:18.021202 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 23:42:18.021202 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 23:42:18.021202 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 23:42:18.021202 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 23:42:18.021202 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 23:42:18.021202 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 23:42:18.021202 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application conn.query(q) Sep 10 23:42:18.021202 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 23:42:18.021202 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 23:42:18.021202 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 23:42:18.021202 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application result.read() Sep 10 23:42:18.021202 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 23:42:18.021202 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 23:42:18.021202 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 23:42:18.021202 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 23:42:18.021988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 23:42:18.021988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 23:42:18.021988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 23:42:18.021988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 23:42:18.021988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 10 23:42:18.021988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 10 23:42:18.021988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [parameters: {'id': 'b978db94120844c5b2171342301d258a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 10 23:42:18.021988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 10 23:42:18.021988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:18.021988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:42:18.021988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:18.021988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:18.021988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:18.021988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:18.021988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:18.021988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:18.024477 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:18.024477 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:18.024477 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:18.024477 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:18.024477 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:18.024477 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:18.024477 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 10 23:42:18.024477 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 10 23:42:18.024477 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:18.024477 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:18.024477 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 10 23:42:18.024477 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 10 23:42:18.024477 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 10 23:42:18.024477 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 10 23:42:18.024477 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 10 23:42:18.024477 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:18.028919 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 245/477] 10.222.0.17 () {60 vars in 1131 bytes} [Sat Sep 10 23:42:17 2022] POST /identity/v3/roles => generated 144 bytes in 248 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:18.028919 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127dba6-dccc-4ef4-a985-f10a078e6e7c None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:18.112578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:18.126761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:18.156808 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbfb3eb-89b1-4cc9-83ba-a1fb4fb6635c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:18.161423 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127dba6-dccc-4ef4-a985-f10a078e6e7c None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:18.202745 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e7a2ac-1807-4abf-8236-9f9fae0ead81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:18.203700 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7a2ac-1807-4abf-8236-9f9fae0ead81 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:18.204417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7a2ac-1807-4abf-8236-9f9fae0ead81 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:18.204827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7a2ac-1807-4abf-8236-9f9fae0ead81 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:18.217703 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e7a2ac-1807-4abf-8236-9f9fae0ead81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:18.223956 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-c4e7a2ac-1807-4abf-8236-9f9fae0ead81 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 23:42:18.223956 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:18.223956 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:18.223956 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:18.223956 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:18.223956 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:18.223956 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:18.223956 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:18.223956 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:18.223956 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:18.223956 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:18.223956 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:18.223956 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:18.223956 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:18.223956 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:18.223956 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:18.225035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:18.225035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:18.225035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:18.225035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:18.225035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:18.225035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:18.225035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:18.225035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:18.225035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:18.225035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:18.225035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:18.225035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:18.225035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:18.225035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:18.225035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:18.225035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:18.225998 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:18.225998 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:18.225998 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:18.225998 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:18.225998 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:18.225998 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:18.225998 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:18.225998 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:18.225998 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:18.225998 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:18.225998 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 23:42:18.225998 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:18.228720 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 246/478] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Sep 10 23:42:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:42:18.236378 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e7a2ac-1807-4abf-8236-9f9fae0ead81 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:18.241945 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa808941-24a3-44a6-8af2-7f63c5afdc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:18.243689 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa808941-24a3-44a6-8af2-7f63c5afdc6d None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:18.244195 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa808941-24a3-44a6-8af2-7f63c5afdc6d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:18.244754 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa808941-24a3-44a6-8af2-7f63c5afdc6d None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:18.245604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa808941-24a3-44a6-8af2-7f63c5afdc6d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:18.250689 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa808941-24a3-44a6-8af2-7f63c5afdc6d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:18.371333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e7a2ac-1807-4abf-8236-9f9fae0ead81 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:18.386466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 233/479] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Sep 10 23:42:17 2022] POST /identity/v3/users => generated 249 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 23:42:18.391327 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:18.412788 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef3035-c2cb-4b31-b9c9-34db1bfc9451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:18.413725 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef3035-c2cb-4b31-b9c9-34db1bfc9451 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:18.414190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef3035-c2cb-4b31-b9c9-34db1bfc9451 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:18.414651 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef3035-c2cb-4b31-b9c9-34db1bfc9451 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:18.422792 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef3035-c2cb-4b31-b9c9-34db1bfc9451 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:18.422792 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef3035-c2cb-4b31-b9c9-34db1bfc9451 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:18.422792 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef3035-c2cb-4b31-b9c9-34db1bfc9451 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:18.435162 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 247/480] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:19.507971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946d7cf-6375-4575-bd4f-1aec1330e9ad None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:19.508891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946d7cf-6375-4575-bd4f-1aec1330e9ad None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:19.509306 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946d7cf-6375-4575-bd4f-1aec1330e9ad None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:19.511126 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 234/481] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:19.516324 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb9947-8cec-45be-8569-6954e94bd9de None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:19.517047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb9947-8cec-45be-8569-6954e94bd9de None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:19.517484 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb9947-8cec-45be-8569-6954e94bd9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:19.541506 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-9acb9947-8cec-45be-8569-6954e94bd9de None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:19.542915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9acb9947-8cec-45be-8569-6954e94bd9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:19.640132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 248/482] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:20.028615 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946d7cf-6375-4575-bd4f-1aec1330e9ad None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:20.114280 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f7fc6-f881-4056-90af-6601d0d469bc None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:20.114982 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f7fc6-f881-4056-90af-6601d0d469bc None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:20.115448 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f7fc6-f881-4056-90af-6601d0d469bc None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:20.116979 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 249/483] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:20.122741 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966d370-20f9-4484-81ee-469442145828 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:20.122741 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966d370-20f9-4484-81ee-469442145828 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:20.122741 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966d370-20f9-4484-81ee-469442145828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:20.165344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b966d370-20f9-4484-81ee-469442145828 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:20.165344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b966d370-20f9-4484-81ee-469442145828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:20.167172 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946d7cf-6375-4575-bd4f-1aec1330e9ad None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:20.215402 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac58bb-87a2-48ad-a2e4-3e187b3a3df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:20.218755 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac58bb-87a2-48ad-a2e4-3e187b3a3df0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:20.219265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac58bb-87a2-48ad-a2e4-3e187b3a3df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:20.219730 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac58bb-87a2-48ad-a2e4-3e187b3a3df0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:20.242702 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac58bb-87a2-48ad-a2e4-3e187b3a3df0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:20.247116 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-feac58bb-87a2-48ad-a2e4-3e187b3a3df0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:20.247116 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:20.247116 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:20.247116 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:20.247116 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:20.247116 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:20.247116 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:20.247116 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:20.247116 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:20.247116 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:20.247116 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:20.247116 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:20.247116 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:20.247116 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:20.247116 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:20.247116 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:20.248057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:20.248057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:20.248057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:20.248057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:20.248057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:20.248057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:20.248057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:20.248057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:20.248057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:20.248057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:20.248057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:20.248057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:20.248057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:20.248057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:20.248057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:20.248057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:20.249449 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:20.249449 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:20.249449 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:20.249449 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:20.249449 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:20.249449 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:20.249449 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:20.249449 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:20.249449 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:20.249449 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:20.249449 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:20.249449 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:20.249449 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 235/484] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Sep 10 23:42:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:20.249449 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:20.255041 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac58bb-87a2-48ad-a2e4-3e187b3a3df0 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:20.279875 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 250/485] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:20.412834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac58bb-87a2-48ad-a2e4-3e187b3a3df0 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:20.463466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd1f8c-f486-4fb3-8b40-e89505c6c265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:20.464465 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd1f8c-f486-4fb3-8b40-e89505c6c265 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:20.464953 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd1f8c-f486-4fb3-8b40-e89505c6c265 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:20.466167 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd1f8c-f486-4fb3-8b40-e89505c6c265 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:20.467180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd1f8c-f486-4fb3-8b40-e89505c6c265 None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:20.467686 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd1f8c-f486-4fb3-8b40-e89505c6c265 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:20.469276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd1f8c-f486-4fb3-8b40-e89505c6c265 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:20.482648 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 236/486] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Sep 10 23:42:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:20.492812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b966d370-20f9-4484-81ee-469442145828 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:20.553629 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:20.561588 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd1f8c-f486-4fb3-8b40-e89505c6c265 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:20.652362 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b966d370-20f9-4484-81ee-469442145828 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:20.689692 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb966187-3b57-430c-abe6-9b09a998f954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:20.690321 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb966187-3b57-430c-abe6-9b09a998f954 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:20.690597 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb966187-3b57-430c-abe6-9b09a998f954 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:20.690908 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb966187-3b57-430c-abe6-9b09a998f954 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:20.693133 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:20.704043 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb966187-3b57-430c-abe6-9b09a998f954 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:20.705497 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb966187-3b57-430c-abe6-9b09a998f954 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:20.721927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-fb966187-3b57-430c-abe6-9b09a998f954 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:20.721927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:20.721927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:20.721927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:20.721927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:20.721927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:20.721927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:20.721927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:20.721927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:20.721927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:20.721927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:20.721927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:20.721927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:20.721927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:20.721927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:20.721927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:20.722855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:20.722855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:20.722855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:20.722855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:20.722855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:20.722855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:20.722855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:20.722855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:20.722855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:20.722855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:20.722855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:20.722855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:20.722855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:20.722855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:20.722855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:20.722855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:20.722855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:20.723986 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:20.723986 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:20.723986 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:20.723986 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 251/487] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:42:20.728719 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb966187-3b57-430c-abe6-9b09a998f954 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:20.763871 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd1f8c-f486-4fb3-8b40-e89505c6c265 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:20.809566 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a26054-b7b1-4dd5-8051-d69642aefacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:20.810738 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a26054-b7b1-4dd5-8051-d69642aefacc None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:20.811240 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a26054-b7b1-4dd5-8051-d69642aefacc None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:20.811720 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a26054-b7b1-4dd5-8051-d69642aefacc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:20.826147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a26054-b7b1-4dd5-8051-d69642aefacc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:20.830966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:20.835245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-c5a26054-b7b1-4dd5-8051-d69642aefacc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:20.835245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:20.835245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:20.835245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:20.835245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:20.835245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:20.835245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:20.835245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:20.835245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:20.835245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:20.835245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:20.835245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:20.835245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:20.835245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:20.835245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:20.835245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:20.836146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:20.836146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:20.836146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:20.836146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:20.836146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:20.836146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:20.836146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:20.836146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:20.836146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:20.836146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:20.836146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:20.836146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:20.836146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:20.836146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:20.836146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:20.836146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:20.836916 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:20.836916 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:20.836916 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:20.836916 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:20.836916 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:20.836916 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:20.836916 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:20.836916 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:20.836916 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:20.836916 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:20.836916 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:20.836916 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:20.839755 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 237/488] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Sep 10 23:42:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:42:20.842250 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3932fb-2f52-4e56-99d9-ee989dfbacc8 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:20.842700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3932fb-2f52-4e56-99d9-ee989dfbacc8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:20.843112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3932fb-2f52-4e56-99d9-ee989dfbacc8 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:20.844121 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb966187-3b57-430c-abe6-9b09a998f954 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:20.844802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 238/489] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:20.849218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3932fb-2f52-4e56-99d9-ee989dfbacc8 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:20.894478 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c7672-e165-4532-b959-f8f3df38c90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:20.895116 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c7672-e165-4532-b959-f8f3df38c90b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:20.895386 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c7672-e165-4532-b959-f8f3df38c90b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:20.895674 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c7672-e165-4532-b959-f8f3df38c90b None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:20.896795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c7672-e165-4532-b959-f8f3df38c90b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:20.897338 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c7672-e165-4532-b959-f8f3df38c90b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:20.899049 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c7672-e165-4532-b959-f8f3df38c90b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:20.915129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 252/490] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:42:20.917075 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2526b5-f907-48d9-9613-fdde75515669 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:20.917355 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2526b5-f907-48d9-9613-fdde75515669 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:20.917624 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2526b5-f907-48d9-9613-fdde75515669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:20.943139 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-4a2526b5-f907-48d9-9613-fdde75515669 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:20.944921 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4a2526b5-f907-48d9-9613-fdde75515669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:20.994054 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3932fb-2f52-4e56-99d9-ee989dfbacc8 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:21.019500 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:21.021349 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 253/491] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:21.031001 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2526b5-f907-48d9-9613-fdde75515669 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:21.040193 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838379d8-0187-4c52-9b90-df5f1b0d1040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:21.040805 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838379d8-0187-4c52-9b90-df5f1b0d1040 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:21.041113 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838379d8-0187-4c52-9b90-df5f1b0d1040 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:21.041472 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838379d8-0187-4c52-9b90-df5f1b0d1040 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:21.042144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838379d8-0187-4c52-9b90-df5f1b0d1040 None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:21.042444 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838379d8-0187-4c52-9b90-df5f1b0d1040 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:21.044035 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838379d8-0187-4c52-9b90-df5f1b0d1040 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:21.061298 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 239/492] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Sep 10 23:42:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:21.062728 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa83e0-e498-49b0-85a2-59423d08951c None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:21.063018 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa83e0-e498-49b0-85a2-59423d08951c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:21.063295 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa83e0-e498-49b0-85a2-59423d08951c None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:21.065065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 240/493] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:21.065770 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa83e0-e498-49b0-85a2-59423d08951c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:21.121141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:21.143471 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2526b5-f907-48d9-9613-fdde75515669 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:21.190480 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d13f87-cad2-473f-b218-9fdfb27640b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:21.191409 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d13f87-cad2-473f-b218-9fdfb27640b5 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:21.191820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d13f87-cad2-473f-b218-9fdfb27640b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:21.192220 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d13f87-cad2-473f-b218-9fdfb27640b5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:21.193984 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa83e0-e498-49b0-85a2-59423d08951c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:21.201104 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d13f87-cad2-473f-b218-9fdfb27640b5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:21.203328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d13f87-cad2-473f-b218-9fdfb27640b5 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:21.220174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-37d13f87-cad2-473f-b218-9fdfb27640b5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 23:42:21.220174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:21.220174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:21.220174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:21.220174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:21.220174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:21.220174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:21.220174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:21.220174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:21.220174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:21.220174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:21.220174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:21.220174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:42:21.220174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:42:21.220174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:42:21.220174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:42:21.221282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:21.221282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:21.221282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:42:21.221282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:21.221282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:42:21.221282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:21.221282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:21.221282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:21.221282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:21.221282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:21.221282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:21.221282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:21.221282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:21.221282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:21.221282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:21.221282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:21.221282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:21.221282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:21.222272 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:21.222272 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:21.222272 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:42:21.222272 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:42:21.222272 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:42:21.222272 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:42:21.222272 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:42:21.222272 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:42:21.222272 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 23:42:21.222272 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:21.224990 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 254/494] 10.222.0.17 () {56 vars in 1093 bytes} [Sat Sep 10 23:42:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:21.231420 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab8371-7d49-4397-8a81-48a9f6105e6b None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:21.231704 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab8371-7d49-4397-8a81-48a9f6105e6b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:21.231983 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab8371-7d49-4397-8a81-48a9f6105e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:21.239585 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ba7a3-fdb2-402f-8671-63db0dc4006e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:21.241204 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ba7a3-fdb2-402f-8671-63db0dc4006e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:21.241622 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ba7a3-fdb2-402f-8671-63db0dc4006e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:21.242027 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ba7a3-fdb2-402f-8671-63db0dc4006e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:21.252198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-e6ab8371-7d49-4397-8a81-48a9f6105e6b None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:21.253201 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e6ab8371-7d49-4397-8a81-48a9f6105e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:21.254415 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337ba7a3-fdb2-402f-8671-63db0dc4006e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:21.256948 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337ba7a3-fdb2-402f-8671-63db0dc4006e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:21.274725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-337ba7a3-fdb2-402f-8671-63db0dc4006e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:21.274725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:21.274725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:21.274725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:21.274725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:21.274725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:21.274725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:21.274725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:21.274725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:21.274725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:21.274725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:21.274725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:21.274725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:21.274725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:21.274725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:21.274725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:21.275747 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:21.275747 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:21.275747 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:21.275747 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:21.275747 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:21.275747 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:21.275747 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:21.275747 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:21.275747 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:21.275747 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:21.275747 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:21.275747 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:21.275747 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:21.275747 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:21.275747 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:21.275747 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:21.275747 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:21.276688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:21.276688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:21.276688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:21.279723 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 241/495] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:21.280632 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ba7a3-fdb2-402f-8671-63db0dc4006e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:21.330245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:21.343098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 255/496] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:21.344162 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab8371-7d49-4397-8a81-48a9f6105e6b None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:21.422193 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ba7a3-fdb2-402f-8671-63db0dc4006e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:21.436555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:21.471176 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f89488-583f-405d-983c-4aee71ba0a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:21.472015 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f89488-583f-405d-983c-4aee71ba0a86 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:21.472534 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f89488-583f-405d-983c-4aee71ba0a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:21.472796 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab8371-7d49-4397-8a81-48a9f6105e6b None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:21.473113 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f89488-583f-405d-983c-4aee71ba0a86 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:21.474053 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f89488-583f-405d-983c-4aee71ba0a86 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:21.474780 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f89488-583f-405d-983c-4aee71ba0a86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:21.477417 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f89488-583f-405d-983c-4aee71ba0a86 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:21.510879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f3c5d-4181-4a20-b269-498693686bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:21.511486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f3c5d-4181-4a20-b269-498693686bba None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:21.511757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f3c5d-4181-4a20-b269-498693686bba None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:21.512591 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f3c5d-4181-4a20-b269-498693686bba None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:21.513261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f3c5d-4181-4a20-b269-498693686bba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:21.513607 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f3c5d-4181-4a20-b269-498693686bba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:21.514994 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f3c5d-4181-4a20-b269-498693686bba None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:21.523303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 242/497] 10.222.0.17 () {56 vars in 1133 bytes} [Sat Sep 10 23:42:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:21.526782 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f89488-583f-405d-983c-4aee71ba0a86 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:21.526929 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 256/498] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:21.528686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b635cac-db07-4ae7-a5db-9a0e2442d16c None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:21.528887 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b635cac-db07-4ae7-a5db-9a0e2442d16c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:21.529174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b635cac-db07-4ae7-a5db-9a0e2442d16c None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:21.532157 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 257/499] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:21.532157 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b635cac-db07-4ae7-a5db-9a0e2442d16c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:21.634766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:21.641169 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f89488-583f-405d-983c-4aee71ba0a86 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:21.644045 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b635cac-db07-4ae7-a5db-9a0e2442d16c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:21.672336 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:21.674847 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54197774-aab8-4893-be59-44f2b3a1b311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:21.675471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54197774-aab8-4893-be59-44f2b3a1b311 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:21.675749 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54197774-aab8-4893-be59-44f2b3a1b311 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:21.676021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54197774-aab8-4893-be59-44f2b3a1b311 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:21.677808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4d039-7b06-4272-9659-9cd8375917f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:21.678409 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4d039-7b06-4272-9659-9cd8375917f3 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:21.678718 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4d039-7b06-4272-9659-9cd8375917f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:21.678990 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4d039-7b06-4272-9659-9cd8375917f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:21.687128 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54197774-aab8-4893-be59-44f2b3a1b311 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:21.691902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-54197774-aab8-4893-be59-44f2b3a1b311 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:21.691902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:21.691902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:21.691902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:21.691902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:21.691902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:21.691902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:21.691902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:21.691902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:21.691902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:21.691902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:21.691902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:21.691902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:21.691902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:21.691902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:21.691902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:21.692970 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:21.692970 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:21.692970 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:21.692970 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:21.692970 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:21.692970 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:21.692970 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:21.692970 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:21.692970 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:21.692970 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:21.692970 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:21.692970 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:21.692970 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:21.692970 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:21.692970 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:21.692970 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:21.699135 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:21.699135 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:21.699135 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:21.699135 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:21.699135 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:21.699135 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:21.699135 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:21.699135 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:21.699135 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:21.699135 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:21.699135 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:21.699135 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:21.699135 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 243/500] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Sep 10 23:42:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:21.699135 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54197774-aab8-4893-be59-44f2b3a1b311 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:21.748229 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e4d039-7b06-4272-9659-9cd8375917f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:21.749660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e4d039-7b06-4272-9659-9cd8375917f3 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:21.792466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54197774-aab8-4893-be59-44f2b3a1b311 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:21.827141 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1d5d7-53a1-44a2-a766-226657aba1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:21.827750 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1d5d7-53a1-44a2-a766-226657aba1cd None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:21.828029 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1d5d7-53a1-44a2-a766-226657aba1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:21.828301 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1d5d7-53a1-44a2-a766-226657aba1cd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:21.835200 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1d5d7-53a1-44a2-a766-226657aba1cd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:21.836656 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1d5d7-53a1-44a2-a766-226657aba1cd None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:21.844551 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 258/501] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:42:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:42:21.845763 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c8e2d-766f-4745-b564-edba5d3068a8 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:21.846043 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c8e2d-766f-4745-b564-edba5d3068a8 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:21.846310 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c8e2d-766f-4745-b564-edba5d3068a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:21.849761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-32c1d5d7-53a1-44a2-a766-226657aba1cd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 23:42:21.849761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:21.849761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:21.849761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:21.849761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:21.849761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:21.849761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:21.849761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:21.849761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:21.849761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:21.849761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:21.849761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:21.849761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:42:21.849761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:42:21.849761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:42:21.849761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:42:21.850609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:21.850609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:21.850609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:42:21.850609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:21.850609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:42:21.850609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:21.850609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:21.850609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:21.850609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:21.850609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:21.850609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:21.850609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:21.850609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:21.850609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:21.850609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:21.850609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:21.850609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:21.850609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:21.852073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:21.852073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:21.852073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:42:21.852073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:42:21.852073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:42:21.852073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:42:21.852073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:42:21.852073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:42:21.852073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 23:42:21.852073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:21.852073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 244/502] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:42:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:21.852073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1d5d7-53a1-44a2-a766-226657aba1cd None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:21.867540 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:21.869367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a39c8e2d-766f-4745-b564-edba5d3068a8 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:21.870557 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a39c8e2d-766f-4745-b564-edba5d3068a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:21.895744 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:21.939906 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 259/503] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:21.940814 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c8e2d-766f-4745-b564-edba5d3068a8 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:21.957787 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1d5d7-53a1-44a2-a766-226657aba1cd None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:21.993213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd2141-eaf9-4dc2-a675-25f34e11d553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:21.993827 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd2141-eaf9-4dc2-a675-25f34e11d553 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:21.994106 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd2141-eaf9-4dc2-a675-25f34e11d553 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:21.994376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd2141-eaf9-4dc2-a675-25f34e11d553 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:22.004389 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbd2141-eaf9-4dc2-a675-25f34e11d553 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:22.005855 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbd2141-eaf9-4dc2-a675-25f34e11d553 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:22.020138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-afbd2141-eaf9-4dc2-a675-25f34e11d553 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:22.020138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:22.020138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:22.020138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:22.020138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:22.020138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:22.020138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:22.020138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:22.020138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:22.020138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:22.020138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:22.020138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:22.020138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:22.020138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:22.020138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:22.020138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:22.021009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:22.021009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:22.021009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:22.021009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:22.021009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:22.021009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:22.021009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:22.021009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:22.021009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:22.021009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:22.021009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:22.021009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:22.021009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:22.021009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:22.021009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:22.021009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:22.021009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:22.021825 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:22.021825 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:22.021825 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:22.021825 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 245/504] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:22.022050 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd2141-eaf9-4dc2-a675-25f34e11d553 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:22.088511 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:22.103363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c8e2d-766f-4745-b564-edba5d3068a8 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:22.122409 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd2141-eaf9-4dc2-a675-25f34e11d553 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:22.140916 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f725a8c-e04f-4f5f-9ec9-a8d23ea2578d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:22.141909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f725a8c-e04f-4f5f-9ec9-a8d23ea2578d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:22.142188 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f725a8c-e04f-4f5f-9ec9-a8d23ea2578d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:22.142456 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f725a8c-e04f-4f5f-9ec9-a8d23ea2578d None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:22.143139 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f725a8c-e04f-4f5f-9ec9-a8d23ea2578d None admin] RBAC: Adding query filter params (name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:22.143440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f725a8c-e04f-4f5f-9ec9-a8d23ea2578d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:22.145260 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f725a8c-e04f-4f5f-9ec9-a8d23ea2578d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:22.158384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 260/505] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Sep 10 23:42:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:22.158384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f725a8c-e04f-4f5f-9ec9-a8d23ea2578d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:22.162460 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b9b9c-a3d0-4814-8f4c-58ad433a7f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:22.163111 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b9b9c-a3d0-4814-8f4c-58ad433a7f43 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:22.163439 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b9b9c-a3d0-4814-8f4c-58ad433a7f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:22.163709 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b9b9c-a3d0-4814-8f4c-58ad433a7f43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:22.175536 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b9b9c-a3d0-4814-8f4c-58ad433a7f43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:22.177087 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b9b9c-a3d0-4814-8f4c-58ad433a7f43 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:22.184319 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:22.191798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-c51b9b9c-a3d0-4814-8f4c-58ad433a7f43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:22.191798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:22.191798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:22.191798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:22.191798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:22.191798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:22.191798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:22.191798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:22.191798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:22.191798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:22.191798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:22.191798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:22.191798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:22.191798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:22.191798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:22.191798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:22.198413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:22.198413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:22.198413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:22.198413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:22.198413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:22.198413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:22.198413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:22.198413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:22.198413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:22.198413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:22.198413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:22.198413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:22.198413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:22.198413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:22.198413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:22.198413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:22.198413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:22.200071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:22.200071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:22.200071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:22.200071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 246/506] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:22.200071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b9b9c-a3d0-4814-8f4c-58ad433a7f43 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:22.270357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f725a8c-e04f-4f5f-9ec9-a8d23ea2578d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:22.294896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b9b9c-a3d0-4814-8f4c-58ad433a7f43 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:22.310173 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9985f-8a16-43dc-9cbc-052bc295d02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:22.310826 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9985f-8a16-43dc-9cbc-052bc295d02d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:22.311102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9985f-8a16-43dc-9cbc-052bc295d02d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:22.311368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9985f-8a16-43dc-9cbc-052bc295d02d None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:22.312086 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9985f-8a16-43dc-9cbc-052bc295d02d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:22.312384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9985f-8a16-43dc-9cbc-052bc295d02d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:22.313799 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9985f-8a16-43dc-9cbc-052bc295d02d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:22.315030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:22.331371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af7c9f-27d9-449d-8372-48f5fd38c72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:22.331966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af7c9f-27d9-449d-8372-48f5fd38c72f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:22.332241 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af7c9f-27d9-449d-8372-48f5fd38c72f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:22.332531 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af7c9f-27d9-449d-8372-48f5fd38c72f None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:22.333217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af7c9f-27d9-449d-8372-48f5fd38c72f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:22.333520 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af7c9f-27d9-449d-8372-48f5fd38c72f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:22.335023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af7c9f-27d9-449d-8372-48f5fd38c72f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:22.348278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 247/507] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:22.349103 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af7c9f-27d9-449d-8372-48f5fd38c72f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:22.355099 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 261/508] 10.222.0.17 () {56 vars in 1137 bytes} [Sat Sep 10 23:42:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:22.355099 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9985f-8a16-43dc-9cbc-052bc295d02d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:22.432246 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:22.450306 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af7c9f-27d9-449d-8372-48f5fd38c72f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:22.451469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9985f-8a16-43dc-9cbc-052bc295d02d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:22.484810 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba58c74-386f-4db9-834c-ce8d3db16bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:22.485427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba58c74-386f-4db9-834c-ce8d3db16bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:22.485706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba58c74-386f-4db9-834c-ce8d3db16bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:22.486033 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba58c74-386f-4db9-834c-ce8d3db16bd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:22.486967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2787301-35f5-4fc3-ad14-a14ff4264e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:22.487565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2787301-35f5-4fc3-ad14-a14ff4264e66 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:22.487847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2787301-35f5-4fc3-ad14-a14ff4264e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:22.488114 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2787301-35f5-4fc3-ad14-a14ff4264e66 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:22.488750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2787301-35f5-4fc3-ad14-a14ff4264e66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:22.489047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2787301-35f5-4fc3-ad14-a14ff4264e66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:22.490420 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2787301-35f5-4fc3-ad14-a14ff4264e66 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:22.496037 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba58c74-386f-4db9-834c-ce8d3db16bd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:22.498991 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba58c74-386f-4db9-834c-ce8d3db16bd2 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:22.502550 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 262/509] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:22.503416 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2787301-35f5-4fc3-ad14-a14ff4264e66 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:22.516208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-bba58c74-386f-4db9-834c-ce8d3db16bd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:22.516208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:22.516208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:22.516208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:22.516208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:22.516208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:22.516208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:22.516208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:22.516208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:22.516208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:22.516208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:22.516208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:22.516208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:22.516208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:22.516208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:22.516208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:22.517281 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:22.517281 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:22.517281 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:22.517281 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:22.517281 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:22.517281 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:22.517281 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:22.517281 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:22.517281 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:22.517281 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:22.517281 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:22.517281 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:22.517281 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:22.517281 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:22.517281 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:22.517281 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:22.517281 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:22.518275 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:22.518275 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:22.518275 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:22.519070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 248/510] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:22.519837 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba58c74-386f-4db9-834c-ce8d3db16bd2 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:22.552372 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:22.607987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2787301-35f5-4fc3-ad14-a14ff4264e66 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:22.620870 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba58c74-386f-4db9-834c-ce8d3db16bd2 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:22.642178 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e6f4e-c4fc-4a0c-a324-d17d997fd9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:22.642836 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e6f4e-c4fc-4a0c-a324-d17d997fd9e3 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:22.643116 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e6f4e-c4fc-4a0c-a324-d17d997fd9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:22.643390 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e6f4e-c4fc-4a0c-a324-d17d997fd9e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:22.650961 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:22.654533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3e6f4e-c4fc-4a0c-a324-d17d997fd9e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:22.655008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19777397-0b9e-4812-bf7c-3a04562ddd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:22.655598 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19777397-0b9e-4812-bf7c-3a04562ddd2f None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:22.655870 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19777397-0b9e-4812-bf7c-3a04562ddd2f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:22.656138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19777397-0b9e-4812-bf7c-3a04562ddd2f None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:22.656774 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19777397-0b9e-4812-bf7c-3a04562ddd2f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:22.658228 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19777397-0b9e-4812-bf7c-3a04562ddd2f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:22.659782 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-7b3e6f4e-c4fc-4a0c-a324-d17d997fd9e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:22.659782 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:22.659782 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:22.659782 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:22.659782 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:22.659782 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:22.659782 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:22.659782 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:22.659782 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:22.659782 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:22.659782 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:22.659782 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:22.659782 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:22.659782 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:22.659782 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:22.659782 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:22.663010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:22.663010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:22.663010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:22.663010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:22.663010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:22.663010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:22.663010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:22.663010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:22.663010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:22.663010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:22.663010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:22.663010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:22.663010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:22.663010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:22.663010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:22.663010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:22.663810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:22.663810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:22.663810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:22.663810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:22.663810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:22.663810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:22.663810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:22.663810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:22.663810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:22.663810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:22.663810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:22.663810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:22.663810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 263/511] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Sep 10 23:42:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:22.663810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e6f4e-c4fc-4a0c-a324-d17d997fd9e3 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:22.737491 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 249/512] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Sep 10 23:42:22 2022] POST /identity/v3/users => generated 247 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:42:22.738416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19777397-0b9e-4812-bf7c-3a04562ddd2f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:22.772439 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e6f4e-c4fc-4a0c-a324-d17d997fd9e3 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:22.812464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e983e24-e24c-4bce-bdd4-c7d75cb26dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:22.813088 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e983e24-e24c-4bce-bdd4-c7d75cb26dce None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:22.813366 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e983e24-e24c-4bce-bdd4-c7d75cb26dce None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:22.813637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e983e24-e24c-4bce-bdd4-c7d75cb26dce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:22.856709 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:22.873413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19777397-0b9e-4812-bf7c-3a04562ddd2f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:22.888807 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:22.891265 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e983e24-e24c-4bce-bdd4-c7d75cb26dce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:22.892842 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e983e24-e24c-4bce-bdd4-c7d75cb26dce None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:22.916088 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747710ba-3535-4979-93d6-c757ab34f6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:22.916718 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747710ba-3535-4979-93d6-c757ab34f6e8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:22.916992 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747710ba-3535-4979-93d6-c757ab34f6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:22.917259 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747710ba-3535-4979-93d6-c757ab34f6e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:22.924414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747710ba-3535-4979-93d6-c757ab34f6e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:22.925881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747710ba-3535-4979-93d6-c757ab34f6e8 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:22.939196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-747710ba-3535-4979-93d6-c757ab34f6e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:22.939196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:22.939196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:22.939196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:22.939196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:22.939196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:22.939196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:22.939196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:22.939196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:22.939196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:22.939196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:22.939196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:22.939196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:42:22.939196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:42:22.939196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:42:22.940065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:42:22.940065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:22.940065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:22.940065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:22.940065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:22.940065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:22.940065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:22.940065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:22.940065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:22.940065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:22.940065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:22.940065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:22.940065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:22.940065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:22.940065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:22.940065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:22.940065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:22.940919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:42:22.940919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:42:22.940919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:42:22.940919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:42:22.940919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:42:22.940919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:42:22.940919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:22.940919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:22.940919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 250/513] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:22.941695 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747710ba-3535-4979-93d6-c757ab34f6e8 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:23.013637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 264/514] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:42:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:42:23.015704 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e983e24-e24c-4bce-bdd4-c7d75cb26dce None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:23.066155 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747710ba-3535-4979-93d6-c757ab34f6e8 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:23.103816 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346936a-2d4c-422a-86b3-15a9c9d34782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:23.104500 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346936a-2d4c-422a-86b3-15a9c9d34782 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:23.104774 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346936a-2d4c-422a-86b3-15a9c9d34782 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:23.105041 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346936a-2d4c-422a-86b3-15a9c9d34782 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:23.105875 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346936a-2d4c-422a-86b3-15a9c9d34782 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:23.106185 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346936a-2d4c-422a-86b3-15a9c9d34782 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:23.107810 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346936a-2d4c-422a-86b3-15a9c9d34782 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:23.124007 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 251/515] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:23.124007 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346936a-2d4c-422a-86b3-15a9c9d34782 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:23.125342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:23.128640 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e983e24-e24c-4bce-bdd4-c7d75cb26dce None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:23.133380 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:23.164442 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980cd3a0-f5d8-4c0f-8393-3960ba442e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:23.165050 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980cd3a0-f5d8-4c0f-8393-3960ba442e1d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:23.165319 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980cd3a0-f5d8-4c0f-8393-3960ba442e1d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:23.165583 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980cd3a0-f5d8-4c0f-8393-3960ba442e1d None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:23.166232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980cd3a0-f5d8-4c0f-8393-3960ba442e1d None admin] RBAC: Adding query filter params (name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:23.166530 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980cd3a0-f5d8-4c0f-8393-3960ba442e1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:23.167940 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980cd3a0-f5d8-4c0f-8393-3960ba442e1d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:23.176963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 265/516] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Sep 10 23:42:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:23.178045 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980cd3a0-f5d8-4c0f-8393-3960ba442e1d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:23.232523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346936a-2d4c-422a-86b3-15a9c9d34782 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:23.267844 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269be24e-bc0e-43d5-b599-7c5de5193c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:23.268360 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269be24e-bc0e-43d5-b599-7c5de5193c07 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:23.268633 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269be24e-bc0e-43d5-b599-7c5de5193c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:23.268898 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269be24e-bc0e-43d5-b599-7c5de5193c07 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:23.269587 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269be24e-bc0e-43d5-b599-7c5de5193c07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:23.269884 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269be24e-bc0e-43d5-b599-7c5de5193c07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:23.271341 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269be24e-bc0e-43d5-b599-7c5de5193c07 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:23.289678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 252/517] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:42:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:23.290514 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269be24e-bc0e-43d5-b599-7c5de5193c07 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:23.313171 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980cd3a0-f5d8-4c0f-8393-3960ba442e1d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:23.348163 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef71a7-87e9-4679-8ad5-d0f34de2a7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:23.348799 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef71a7-87e9-4679-8ad5-d0f34de2a7cd None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:23.349074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef71a7-87e9-4679-8ad5-d0f34de2a7cd None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:23.349344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef71a7-87e9-4679-8ad5-d0f34de2a7cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:23.361137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eef71a7-87e9-4679-8ad5-d0f34de2a7cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:23.362572 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eef71a7-87e9-4679-8ad5-d0f34de2a7cd None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:23.365341 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:23.377966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-5eef71a7-87e9-4679-8ad5-d0f34de2a7cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:23.377966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:23.377966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:23.377966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:23.377966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:23.377966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:23.377966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:23.377966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:23.377966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:23.377966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:23.377966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:23.377966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:23.377966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:23.377966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:23.377966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:23.377966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:23.380753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:23.380753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:23.380753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:23.380753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:23.380753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:23.380753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:23.380753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:23.380753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:23.380753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:23.380753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:23.380753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:23.380753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:23.380753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:23.380753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:23.380753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:23.380753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:23.380753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:23.381593 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:23.381593 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:23.381593 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:23.381593 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 266/518] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:23.381593 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef71a7-87e9-4679-8ad5-d0f34de2a7cd None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:23.398131 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:23.408325 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269be24e-bc0e-43d5-b599-7c5de5193c07 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:23.444291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe33c11-a9d5-4510-a34c-31fcc78e41d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:23.444915 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe33c11-a9d5-4510-a34c-31fcc78e41d3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:23.445190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe33c11-a9d5-4510-a34c-31fcc78e41d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:23.445457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe33c11-a9d5-4510-a34c-31fcc78e41d3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:23.453431 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe33c11-a9d5-4510-a34c-31fcc78e41d3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:23.454935 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe33c11-a9d5-4510-a34c-31fcc78e41d3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:23.467205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-cfe33c11-a9d5-4510-a34c-31fcc78e41d3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 10 23:42:23.467205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:23.467205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:23.467205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:23.467205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:23.467205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:23.467205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:23.467205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:23.467205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:23.467205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:23.467205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:23.467205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:23.467205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:42:23.467205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:42:23.467205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:42:23.467205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:42:23.468102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:23.468102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:23.468102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:42:23.468102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:23.468102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:42:23.468102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:23.468102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:23.468102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:23.468102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:23.468102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:23.468102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:23.468102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:23.468102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:23.468102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:23.468102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:23.468102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:23.468102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:23.468102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:23.468981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:23.468981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:23.468981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:42:23.468981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:42:23.468981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:42:23.468981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:42:23.468981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:42:23.468981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:42:23.468981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 10 23:42:23.468981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:23.468981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 253/519] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:42:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:23.469527 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe33c11-a9d5-4510-a34c-31fcc78e41d3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:23.497252 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef71a7-87e9-4679-8ad5-d0f34de2a7cd None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:23.537089 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b066d-af12-4326-bb84-7123870d28ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:23.537627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b066d-af12-4326-bb84-7123870d28ac None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:23.537913 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b066d-af12-4326-bb84-7123870d28ac None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:23.538163 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b066d-af12-4326-bb84-7123870d28ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:23.549151 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b066d-af12-4326-bb84-7123870d28ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:23.550610 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b066d-af12-4326-bb84-7123870d28ac None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:23.566159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-841b066d-af12-4326-bb84-7123870d28ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:23.566159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:23.566159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:23.566159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:23.566159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:23.566159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:23.566159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:23.566159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:23.566159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:23.566159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:23.566159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:23.566159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:23.566159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:23.566159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:23.566159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:23.566159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:23.567105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:23.567105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:23.567105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:23.567105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:23.567105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:23.567105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:23.567105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:23.567105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:23.567105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:23.567105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:23.567105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:23.567105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:23.567105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:23.567105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:23.567105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:23.567105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:23.567105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:23.567932 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:23.567932 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:23.567932 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:23.567932 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 267/520] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:23.568138 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b066d-af12-4326-bb84-7123870d28ac None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:23.584465 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe33c11-a9d5-4510-a34c-31fcc78e41d3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:23.614502 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:23.624660 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59340d3-bad8-48c9-a1c5-a614659491cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:23.625253 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59340d3-bad8-48c9-a1c5-a614659491cb None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:23.625527 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59340d3-bad8-48c9-a1c5-a614659491cb None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:23.631954 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59340d3-bad8-48c9-a1c5-a614659491cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:23.631954 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59340d3-bad8-48c9-a1c5-a614659491cb None admin] RBAC: Adding query filter params (role.id=8ff1727b65104e5fb97333d477196d46, scope.project.id=947277c4458645899287e6818947c076, user.id=636bfafb693044d283530063047fc0c2) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:23.631954 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59340d3-bad8-48c9-a1c5-a614659491cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:23.631954 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59340d3-bad8-48c9-a1c5-a614659491cb None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:23.634900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 254/521] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:42:23 2022] GET /identity/v3/role_assignments?user.id=636bfafb693044d283530063047fc0c2&scope.project.id=947277c4458645899287e6818947c076&role.id=8ff1727b65104e5fb97333d477196d46 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:23.636080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59340d3-bad8-48c9-a1c5-a614659491cb None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:23.652481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:23.690360 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b066d-af12-4326-bb84-7123870d28ac None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:23.733767 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc3cef-b0b0-4495-be73-ca7c323fec91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:23.734383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc3cef-b0b0-4495-be73-ca7c323fec91 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:23.734796 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc3cef-b0b0-4495-be73-ca7c323fec91 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:23.735067 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc3cef-b0b0-4495-be73-ca7c323fec91 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:23.735715 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc3cef-b0b0-4495-be73-ca7c323fec91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:23.736104 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc3cef-b0b0-4495-be73-ca7c323fec91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:23.737761 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc3cef-b0b0-4495-be73-ca7c323fec91 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:23.751987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 268/522] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:23.752944 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc3cef-b0b0-4495-be73-ca7c323fec91 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:23.774721 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59340d3-bad8-48c9-a1c5-a614659491cb None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:23.821108 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8df4e6-b74d-4c83-bc80-cdf6066e416a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:23.821737 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8df4e6-b74d-4c83-bc80-cdf6066e416a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:23.822084 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8df4e6-b74d-4c83-bc80-cdf6066e416a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:23.822420 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8df4e6-b74d-4c83-bc80-cdf6066e416a None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:23.823860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8df4e6-b74d-4c83-bc80-cdf6066e416a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:23.824240 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8df4e6-b74d-4c83-bc80-cdf6066e416a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:23.826679 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8df4e6-b74d-4c83-bc80-cdf6066e416a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:23.852070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 255/523] 10.222.0.17 () {56 vars in 1137 bytes} [Sat Sep 10 23:42:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:23.853270 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8df4e6-b74d-4c83-bc80-cdf6066e416a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:23.880118 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc3cef-b0b0-4495-be73-ca7c323fec91 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:23.907204 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:23.908013 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:23.916125 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a588f5-92f6-4af4-ac02-e49829ee20e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:23.916976 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a588f5-92f6-4af4-ac02-e49829ee20e4 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:23.922792 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a588f5-92f6-4af4-ac02-e49829ee20e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:23.922792 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a588f5-92f6-4af4-ac02-e49829ee20e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:23.922792 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a588f5-92f6-4af4-ac02-e49829ee20e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:23.922792 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a588f5-92f6-4af4-ac02-e49829ee20e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:23.922792 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a588f5-92f6-4af4-ac02-e49829ee20e4 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:23.932654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 269/524] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:23.933389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a588f5-92f6-4af4-ac02-e49829ee20e4 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:23.967158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8df4e6-b74d-4c83-bc80-cdf6066e416a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:24.010805 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c2f342-8a53-4192-9539-f59d9ceff2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:24.011895 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2f342-8a53-4192-9539-f59d9ceff2b6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:24.012879 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2f342-8a53-4192-9539-f59d9ceff2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:24.013347 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2f342-8a53-4192-9539-f59d9ceff2b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:24.023231 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c2f342-8a53-4192-9539-f59d9ceff2b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:24.025495 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c2f342-8a53-4192-9539-f59d9ceff2b6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:24.044081 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a588f5-92f6-4af4-ac02-e49829ee20e4 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:24.045278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-90c2f342-8a53-4192-9539-f59d9ceff2b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:24.045278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:24.045278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:24.045278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:24.045278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:24.045278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:24.045278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:24.045278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:24.045278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:24.045278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:24.045278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:24.045278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:24.045278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:42:24.045278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:42:24.045278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:42:24.046078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:42:24.046078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:24.046078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:24.046078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:24.046078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:24.046078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:24.046078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:24.046078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:24.046078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:24.046078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:24.046078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:24.046078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:24.046078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:24.046078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:24.046078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:24.046078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:24.046078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:24.046963 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:42:24.046963 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:42:24.046963 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:42:24.046963 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:42:24.046963 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:42:24.046963 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:42:24.046963 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:24.046963 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:24.048013 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 256/525] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:24.053616 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c2f342-8a53-4192-9539-f59d9ceff2b6 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:24.091759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4520207-fb6c-4c09-b9bd-209fb9a635cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:24.092348 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4520207-fb6c-4c09-b9bd-209fb9a635cf None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:24.092629 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4520207-fb6c-4c09-b9bd-209fb9a635cf None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:24.092952 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4520207-fb6c-4c09-b9bd-209fb9a635cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:24.165568 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:24.180516 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c2f342-8a53-4192-9539-f59d9ceff2b6 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:24.180785 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4520207-fb6c-4c09-b9bd-209fb9a635cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:24.183229 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4520207-fb6c-4c09-b9bd-209fb9a635cf None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:24.191519 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:24.227928 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dde721-33af-4f57-ab3c-60aa878a116c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:24.228646 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dde721-33af-4f57-ab3c-60aa878a116c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:24.228991 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dde721-33af-4f57-ab3c-60aa878a116c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:24.229766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dde721-33af-4f57-ab3c-60aa878a116c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:24.239653 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dde721-33af-4f57-ab3c-60aa878a116c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:24.241949 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dde721-33af-4f57-ab3c-60aa878a116c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:24.257314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-19dde721-33af-4f57-ab3c-60aa878a116c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 10 23:42:24.257314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:24.257314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:24.257314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:24.257314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:24.257314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:24.257314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:24.257314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:24.257314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:24.257314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:24.257314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:24.257314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:24.257314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:42:24.257314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:42:24.257314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:42:24.257314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:42:24.258440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:24.258440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:24.258440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:42:24.258440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:24.258440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:42:24.258440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:24.258440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:24.258440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:24.258440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:24.258440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:24.258440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:24.258440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:24.258440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:24.258440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:24.258440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:24.258440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:24.258440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:24.258440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:24.262433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:24.262433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:24.262433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:42:24.262433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:42:24.262433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:42:24.262433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:42:24.262433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:42:24.262433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:42:24.262433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 10 23:42:24.262433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:24.262433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 257/526] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Sep 10 23:42:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:24.262433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dde721-33af-4f57-ab3c-60aa878a116c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:24.320394 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 270/527] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:42:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:42:24.321449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4520207-fb6c-4c09-b9bd-209fb9a635cf None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:24.408321 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dde721-33af-4f57-ab3c-60aa878a116c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:24.413407 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:24.429329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4520207-fb6c-4c09-b9bd-209fb9a635cf None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:24.447740 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe394fd-20c5-4a43-8dae-ffd85096d715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:24.449692 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe394fd-20c5-4a43-8dae-ffd85096d715 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:24.450121 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe394fd-20c5-4a43-8dae-ffd85096d715 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:24.450568 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe394fd-20c5-4a43-8dae-ffd85096d715 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:24.451423 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe394fd-20c5-4a43-8dae-ffd85096d715 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:24.451873 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe394fd-20c5-4a43-8dae-ffd85096d715 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:24.454525 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe394fd-20c5-4a43-8dae-ffd85096d715 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:24.468328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348634eb-73aa-42ae-9cb6-3509d6769b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:24.469352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348634eb-73aa-42ae-9cb6-3509d6769b88 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:24.469732 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348634eb-73aa-42ae-9cb6-3509d6769b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:24.470101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348634eb-73aa-42ae-9cb6-3509d6769b88 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:24.470977 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348634eb-73aa-42ae-9cb6-3509d6769b88 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:24.471518 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348634eb-73aa-42ae-9cb6-3509d6769b88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:24.471689 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 258/528] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:42:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:24.472637 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe394fd-20c5-4a43-8dae-ffd85096d715 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:24.477823 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348634eb-73aa-42ae-9cb6-3509d6769b88 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:24.500676 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 271/529] 10.222.0.17 () {56 vars in 1139 bytes} [Sat Sep 10 23:42:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:24.501820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348634eb-73aa-42ae-9cb6-3509d6769b88 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:24.508639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:24.595358 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe394fd-20c5-4a43-8dae-ffd85096d715 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:24.645292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348634eb-73aa-42ae-9cb6-3509d6769b88 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:24.646498 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b9987-d5fe-4a62-93de-7e5dae467480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:24.647275 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b9987-d5fe-4a62-93de-7e5dae467480 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:24.647655 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b9987-d5fe-4a62-93de-7e5dae467480 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:24.648006 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b9987-d5fe-4a62-93de-7e5dae467480 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:24.659008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b9987-d5fe-4a62-93de-7e5dae467480 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:24.660592 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b9987-d5fe-4a62-93de-7e5dae467480 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:24.676463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-a06b9987-d5fe-4a62-93de-7e5dae467480 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:24.676463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:24.676463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:24.676463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:24.676463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:24.676463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:24.676463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:24.676463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:24.676463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:24.676463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:24.676463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:24.676463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:24.676463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:24.676463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:24.676463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:24.676463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:24.677605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:24.677605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:24.677605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:24.677605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:24.677605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:24.677605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:24.677605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:24.677605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:24.677605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:24.677605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:24.677605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:24.677605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:24.677605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:24.677605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:24.677605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:24.677605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:24.677605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:24.678680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:24.678680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:24.678680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:24.679766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 259/530] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:24.680453 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b9987-d5fe-4a62-93de-7e5dae467480 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:24.690846 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc6996-0f32-441d-9710-a3169ddcbbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:24.692084 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc6996-0f32-441d-9710-a3169ddcbbd0 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:24.692619 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc6996-0f32-441d-9710-a3169ddcbbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:24.693124 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc6996-0f32-441d-9710-a3169ddcbbd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:24.694089 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc6996-0f32-441d-9710-a3169ddcbbd0 None admin] RBAC: Adding query filter params (role.id=8ff1727b65104e5fb97333d477196d46, scope.project.id=947277c4458645899287e6818947c076, user.id=c6fccf47e1ec4324983c5218ec3c96a4) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:24.694692 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc6996-0f32-441d-9710-a3169ddcbbd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:24.696290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc6996-0f32-441d-9710-a3169ddcbbd0 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:24.703622 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 272/531] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:42:24 2022] GET /identity/v3/role_assignments?user.id=c6fccf47e1ec4324983c5218ec3c96a4&scope.project.id=947277c4458645899287e6818947c076&role.id=8ff1727b65104e5fb97333d477196d46 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:24.704604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc6996-0f32-441d-9710-a3169ddcbbd0 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:24.716898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:24.833262 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:24.864944 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc6996-0f32-441d-9710-a3169ddcbbd0 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:24.869475 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b9987-d5fe-4a62-93de-7e5dae467480 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:24.941129 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec2c2a-9549-4215-be63-55c187763586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:24.941755 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec2c2a-9549-4215-be63-55c187763586 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:24.942039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec2c2a-9549-4215-be63-55c187763586 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:24.942291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec2c2a-9549-4215-be63-55c187763586 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:24.942968 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220889fb-88d5-49ee-ae63-7aa29c2b0bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:24.943626 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220889fb-88d5-49ee-ae63-7aa29c2b0bdf None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:24.943895 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220889fb-88d5-49ee-ae63-7aa29c2b0bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:24.944216 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220889fb-88d5-49ee-ae63-7aa29c2b0bdf None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:24.944875 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220889fb-88d5-49ee-ae63-7aa29c2b0bdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:24.945176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220889fb-88d5-49ee-ae63-7aa29c2b0bdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:24.946584 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220889fb-88d5-49ee-ae63-7aa29c2b0bdf None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:24.975436 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 273/532] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:24.982107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220889fb-88d5-49ee-ae63-7aa29c2b0bdf None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:25.057163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec2c2a-9549-4215-be63-55c187763586 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:25.058557 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec2c2a-9549-4215-be63-55c187763586 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:25.072071 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:25.115293 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220889fb-88d5-49ee-ae63-7aa29c2b0bdf None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:25.141432 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:25.158317 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b4be4-2cd3-49c6-b583-684fcc493c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:25.159681 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b4be4-2cd3-49c6-b583-684fcc493c07 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:25.160213 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b4be4-2cd3-49c6-b583-684fcc493c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:25.160629 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b4be4-2cd3-49c6-b583-684fcc493c07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:25.162728 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 260/533] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:42:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:42:25.164104 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a49b7-6202-4a06-a433-cc70cc4febce None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:25.164383 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a49b7-6202-4a06-a433-cc70cc4febce None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:25.164655 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a49b7-6202-4a06-a433-cc70cc4febce None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:25.169698 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b4be4-2cd3-49c6-b583-684fcc493c07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:25.170650 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 261/534] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:25.171507 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a49b7-6202-4a06-a433-cc70cc4febce None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:25.172043 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b4be4-2cd3-49c6-b583-684fcc493c07 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:25.187258 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-b52b4be4-2cd3-49c6-b583-684fcc493c07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:25.187258 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:25.187258 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:25.187258 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:25.187258 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:25.187258 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:25.187258 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:25.187258 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:25.187258 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:25.187258 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:25.187258 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:25.187258 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:25.187258 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:42:25.187258 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:42:25.187258 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:42:25.188363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:42:25.188363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:25.188363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:25.188363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:25.188363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:25.188363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:25.188363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:25.188363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:25.188363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:25.188363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:25.188363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:25.188363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:25.188363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:25.188363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:25.188363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:25.188363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:25.188363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:25.189405 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:42:25.189405 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:42:25.189405 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:42:25.189405 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:42:25.189405 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:42:25.189405 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:42:25.189405 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:25.189405 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:25.192269 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 274/535] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:25.193567 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1285e-6ea7-4422-a38b-52214940c09e None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:25.193972 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1285e-6ea7-4422-a38b-52214940c09e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:25.194391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1285e-6ea7-4422-a38b-52214940c09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:25.215498 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-32c1285e-6ea7-4422-a38b-52214940c09e None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:25.217119 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-32c1285e-6ea7-4422-a38b-52214940c09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:25.290943 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a49b7-6202-4a06-a433-cc70cc4febce None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:25.307439 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 275/536] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:25.308451 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1285e-6ea7-4422-a38b-52214940c09e None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:25.375364 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e987394-1037-422f-beff-cfce61efbb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:25.375995 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e987394-1037-422f-beff-cfce61efbb0a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:25.376175 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:25.376280 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e987394-1037-422f-beff-cfce61efbb0a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:25.376517 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e987394-1037-422f-beff-cfce61efbb0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:25.403817 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e987394-1037-422f-beff-cfce61efbb0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:25.405304 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e987394-1037-422f-beff-cfce61efbb0a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:25.421833 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1285e-6ea7-4422-a38b-52214940c09e None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:25.432629 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-6e987394-1037-422f-beff-cfce61efbb0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:25.432629 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:25.432629 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:25.432629 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:25.432629 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:25.432629 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:25.432629 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:25.432629 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:25.432629 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:25.432629 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:25.432629 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:25.432629 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:25.432629 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:25.432629 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:25.432629 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:25.432629 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:25.433563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:25.433563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:25.433563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:25.433563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:25.433563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:25.433563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:25.433563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:25.433563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:25.433563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:25.433563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:25.433563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:25.433563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:25.433563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:25.433563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:25.433563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:25.433563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:25.433563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:25.435151 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:25.435151 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:25.435151 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:25.443184 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 262/537] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:25.448614 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e987394-1037-422f-beff-cfce61efbb0a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:25.460275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca23fcf-6063-44ed-bebb-8953cb66966a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:25.460885 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca23fcf-6063-44ed-bebb-8953cb66966a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:25.461159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca23fcf-6063-44ed-bebb-8953cb66966a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:25.461477 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca23fcf-6063-44ed-bebb-8953cb66966a None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:25.462221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca23fcf-6063-44ed-bebb-8953cb66966a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:25.462527 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca23fcf-6063-44ed-bebb-8953cb66966a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:25.464173 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca23fcf-6063-44ed-bebb-8953cb66966a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:25.493420 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 276/538] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:42:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:25.500955 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca23fcf-6063-44ed-bebb-8953cb66966a None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:25.512728 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:25.611485 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e987394-1037-422f-beff-cfce61efbb0a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:25.649886 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca23fcf-6063-44ed-bebb-8953cb66966a None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:25.653918 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f64534-3106-4bbc-a8d3-5b110570b84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:25.654517 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f64534-3106-4bbc-a8d3-5b110570b84d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:25.654896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f64534-3106-4bbc-a8d3-5b110570b84d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:25.655174 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f64534-3106-4bbc-a8d3-5b110570b84d None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:25.656780 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f64534-3106-4bbc-a8d3-5b110570b84d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:25.657086 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f64534-3106-4bbc-a8d3-5b110570b84d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:25.658500 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f64534-3106-4bbc-a8d3-5b110570b84d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:25.672618 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 263/539] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:42:25.673475 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f64534-3106-4bbc-a8d3-5b110570b84d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:25.682840 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:25.704154 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac930e-235e-4809-9f01-a6f82b84cfea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:25.704748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac930e-235e-4809-9f01-a6f82b84cfea None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:25.705026 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac930e-235e-4809-9f01-a6f82b84cfea None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:25.705357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac930e-235e-4809-9f01-a6f82b84cfea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:25.706098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ac930e-235e-4809-9f01-a6f82b84cfea None admin] RBAC: Adding query filter params (role.id=8ff1727b65104e5fb97333d477196d46, scope.project.id=947277c4458645899287e6818947c076, user.id=2cb18199dc8b42589184042556e98983) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:25.706419 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ac930e-235e-4809-9f01-a6f82b84cfea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:25.708954 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ac930e-235e-4809-9f01-a6f82b84cfea None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:25.717021 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 277/540] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:42:25 2022] GET /identity/v3/role_assignments?user.id=2cb18199dc8b42589184042556e98983&scope.project.id=947277c4458645899287e6818947c076&role.id=8ff1727b65104e5fb97333d477196d46 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:42:25.718159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac930e-235e-4809-9f01-a6f82b84cfea None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:25.796297 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:25.809837 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f64534-3106-4bbc-a8d3-5b110570b84d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:25.833743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac930e-235e-4809-9f01-a6f82b84cfea None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:25.853142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91068d15-4015-41c7-9540-0a0d18f01a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:25.853782 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91068d15-4015-41c7-9540-0a0d18f01a24 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:25.854065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91068d15-4015-41c7-9540-0a0d18f01a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:25.854338 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91068d15-4015-41c7-9540-0a0d18f01a24 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:25.866390 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91068d15-4015-41c7-9540-0a0d18f01a24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:25.871644 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-91068d15-4015-41c7-9540-0a0d18f01a24 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:25.871644 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:25.871644 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:25.871644 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:25.871644 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:25.871644 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:25.871644 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:25.871644 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:25.871644 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:25.871644 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:25.871644 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:25.871644 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:25.871644 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:25.871644 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:25.871644 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:25.871644 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:25.872599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:25.872599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:25.872599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:25.872599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:25.872599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:25.872599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:25.872599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:25.872599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:25.872599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:25.872599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:25.872599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:25.872599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:25.872599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:25.872599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:25.872599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:25.872599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:25.873371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:25.873371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:25.873371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:25.873371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:25.873371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:25.873371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:25.873371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:25.873371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:25.873371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:25.873371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:25.873371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:25.873371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:25.873371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 264/541] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Sep 10 23:42:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:25.874552 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458d7ef-fc54-4cd2-be50-a1ac7b411c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:25.875376 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458d7ef-fc54-4cd2-be50-a1ac7b411c3b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:25.875777 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458d7ef-fc54-4cd2-be50-a1ac7b411c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:25.876181 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458d7ef-fc54-4cd2-be50-a1ac7b411c3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:25.878704 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91068d15-4015-41c7-9540-0a0d18f01a24 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:25.884619 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458d7ef-fc54-4cd2-be50-a1ac7b411c3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:25.886471 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458d7ef-fc54-4cd2-be50-a1ac7b411c3b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:25.900465 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-3458d7ef-fc54-4cd2-be50-a1ac7b411c3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:25.900465 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:25.900465 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:25.900465 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:25.900465 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:25.900465 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:25.900465 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:25.900465 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:25.900465 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:25.900465 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:25.900465 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:25.900465 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:25.900465 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:42:25.900465 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:42:25.900465 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:42:25.901559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:42:25.901559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:25.901559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:25.901559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:25.901559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:25.901559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:25.901559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:25.901559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:25.901559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:25.901559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:25.901559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:25.901559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:25.901559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:25.901559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:25.901559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:25.901559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:25.901559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:25.902702 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:42:25.902702 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:42:25.902702 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:42:25.902702 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:42:25.902702 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:42:25.902702 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:42:25.902702 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:25.902702 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:25.904876 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 278/542] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:25.911550 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458d7ef-fc54-4cd2-be50-a1ac7b411c3b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:25.976941 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:26.027914 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458d7ef-fc54-4cd2-be50-a1ac7b411c3b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:26.068359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91068d15-4015-41c7-9540-0a0d18f01a24 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:26.086693 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f9ee0-8274-4dc9-b029-e29b17827afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:26.087407 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f9ee0-8274-4dc9-b029-e29b17827afb None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:26.087891 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f9ee0-8274-4dc9-b029-e29b17827afb None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:26.088176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f9ee0-8274-4dc9-b029-e29b17827afb None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:26.089231 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f9ee0-8274-4dc9-b029-e29b17827afb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:26.089544 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f9ee0-8274-4dc9-b029-e29b17827afb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:26.091258 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f9ee0-8274-4dc9-b029-e29b17827afb None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:26.106059 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47f2c4-7a83-4368-a25f-5fc803503f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:26.106573 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 279/543] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:42:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:42:26.106911 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47f2c4-7a83-4368-a25f-5fc803503f45 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:26.107346 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47f2c4-7a83-4368-a25f-5fc803503f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:26.107703 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47f2c4-7a83-4368-a25f-5fc803503f45 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:26.108057 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5703236-02e1-439e-a838-382b78de54b8 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:26.108340 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5703236-02e1-439e-a838-382b78de54b8 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:26.108654 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe47f2c4-7a83-4368-a25f-5fc803503f45 None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:26.108763 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5703236-02e1-439e-a838-382b78de54b8 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:26.109114 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe47f2c4-7a83-4368-a25f-5fc803503f45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:26.110007 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 280/544] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:26.110695 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe47f2c4-7a83-4368-a25f-5fc803503f45 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:26.114773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5703236-02e1-439e-a838-382b78de54b8 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:26.122381 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 265/545] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Sep 10 23:42:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:26.128350 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8839c6e-145f-45d9-993b-60fb16f3c24c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:26.128770 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8839c6e-145f-45d9-993b-60fb16f3c24c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:26.129134 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8839c6e-145f-45d9-993b-60fb16f3c24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:26.149371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d8839c6e-145f-45d9-993b-60fb16f3c24c None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:26.150796 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d8839c6e-145f-45d9-993b-60fb16f3c24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:26.163118 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:26.226690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 266/546] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:26.227897 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8839c6e-145f-45d9-993b-60fb16f3c24c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:26.252363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5703236-02e1-439e-a838-382b78de54b8 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:26.253883 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:26.323931 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5838b-a132-455a-983e-3adb62b8c995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:26.324609 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5838b-a132-455a-983e-3adb62b8c995 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:26.324886 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5838b-a132-455a-983e-3adb62b8c995 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:26.325152 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5838b-a132-455a-983e-3adb62b8c995 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:26.325875 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c5838b-a132-455a-983e-3adb62b8c995 None admin] RBAC: Adding query filter params (role.id=8ff1727b65104e5fb97333d477196d46, scope.project.id=947277c4458645899287e6818947c076, user.id=53e79d9fb2cd452d8d999a84b6425959) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:26.326175 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c5838b-a132-455a-983e-3adb62b8c995 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:26.330128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c5838b-a132-455a-983e-3adb62b8c995 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:26.339115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 281/547] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:42:26 2022] GET /identity/v3/role_assignments?user.id=53e79d9fb2cd452d8d999a84b6425959&scope.project.id=947277c4458645899287e6818947c076&role.id=8ff1727b65104e5fb97333d477196d46 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:26.444946 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8839c6e-145f-45d9-993b-60fb16f3c24c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:26.501987 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325958b0-4b71-4893-8c40-02d3b49fbd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:26.503042 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325958b0-4b71-4893-8c40-02d3b49fbd4d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:26.503536 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325958b0-4b71-4893-8c40-02d3b49fbd4d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:26.505295 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325958b0-4b71-4893-8c40-02d3b49fbd4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:26.521904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325958b0-4b71-4893-8c40-02d3b49fbd4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:26.523715 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325958b0-4b71-4893-8c40-02d3b49fbd4d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:26.526554 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:26.544144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-325958b0-4b71-4893-8c40-02d3b49fbd4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:26.544144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:26.544144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:26.544144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:26.544144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:26.544144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:26.544144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:26.544144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:26.544144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:26.544144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:26.544144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:26.544144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:26.544144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:26.544144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:26.544144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:26.544144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:26.545074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:26.545074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:26.545074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:26.545074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:26.545074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:26.545074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:26.545074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:26.545074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:26.545074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:26.545074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:26.545074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:26.545074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:26.545074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:26.545074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:26.545074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:26.545074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:26.545074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:26.547130 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:26.547130 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:26.547130 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:26.547130 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 267/548] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:26.551701 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5838b-a132-455a-983e-3adb62b8c995 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:26.557627 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325958b0-4b71-4893-8c40-02d3b49fbd4d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:26.715709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5838b-a132-455a-983e-3adb62b8c995 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:26.748839 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325958b0-4b71-4893-8c40-02d3b49fbd4d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:26.755039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee615766-0cc1-482a-a734-ff8a4bc7dac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:26.756604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee615766-0cc1-482a-a734-ff8a4bc7dac3 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:26.756887 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee615766-0cc1-482a-a734-ff8a4bc7dac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:26.757145 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee615766-0cc1-482a-a734-ff8a4bc7dac3 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:26.757797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee615766-0cc1-482a-a734-ff8a4bc7dac3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:26.758094 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee615766-0cc1-482a-a734-ff8a4bc7dac3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:26.759496 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee615766-0cc1-482a-a734-ff8a4bc7dac3 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:26.779376 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 282/549] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:26.785330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee615766-0cc1-482a-a734-ff8a4bc7dac3 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:26.826544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82862a27-1efe-4dbe-92dc-99ebd9f27213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:26.827187 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82862a27-1efe-4dbe-92dc-99ebd9f27213 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:26.827459 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82862a27-1efe-4dbe-92dc-99ebd9f27213 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:26.827722 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82862a27-1efe-4dbe-92dc-99ebd9f27213 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:26.839033 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:26.846975 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82862a27-1efe-4dbe-92dc-99ebd9f27213 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:26.852123 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-82862a27-1efe-4dbe-92dc-99ebd9f27213 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:26.852123 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:26.852123 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:26.852123 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:26.852123 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:26.852123 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:26.852123 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:26.852123 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:26.852123 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:26.852123 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:26.852123 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:26.852123 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:26.852123 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:26.852123 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:26.852123 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:26.852123 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:26.853039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:26.853039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:26.853039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:26.853039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:26.853039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:26.853039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:26.853039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:26.853039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:26.853039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:26.853039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:26.853039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:26.853039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:26.853039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:26.853039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:26.853039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:26.853039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:26.854628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:26.854628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:26.854628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:26.854628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:26.854628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:26.854628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:26.854628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:26.854628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:26.854628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:26.854628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:26.854628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:26.854628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:26.854628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 268/550] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Sep 10 23:42:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:26.867707 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82862a27-1efe-4dbe-92dc-99ebd9f27213 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:26.960086 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:26.969624 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee615766-0cc1-482a-a734-ff8a4bc7dac3 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:27.035765 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea8a22-0d0e-405f-9a38-504813c57ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:27.036314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea8a22-0d0e-405f-9a38-504813c57ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:27.036591 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea8a22-0d0e-405f-9a38-504813c57ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:27.036858 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea8a22-0d0e-405f-9a38-504813c57ab4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:27.039208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82862a27-1efe-4dbe-92dc-99ebd9f27213 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:27.062082 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ea8a22-0d0e-405f-9a38-504813c57ab4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:27.063589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ea8a22-0d0e-405f-9a38-504813c57ab4 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:27.080924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1493d08-3a8b-40a3-a24b-5f623eef07ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:27.081523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1493d08-3a8b-40a3-a24b-5f623eef07ca None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:27.081803 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1493d08-3a8b-40a3-a24b-5f623eef07ca None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:27.082075 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1493d08-3a8b-40a3-a24b-5f623eef07ca None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:27.082775 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1493d08-3a8b-40a3-a24b-5f623eef07ca None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:27.083130 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1493d08-3a8b-40a3-a24b-5f623eef07ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:27.084232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-73ea8a22-0d0e-405f-9a38-504813c57ab4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 10 23:42:27.084232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:27.084232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:27.084232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:27.084232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:27.084232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:27.084232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:27.084232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:27.084232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:27.084232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:27.084232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:27.084232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:27.084232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:42:27.084232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:42:27.084232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:42:27.084232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:42:27.085107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:27.085107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:27.085107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:42:27.085107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:27.085107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:42:27.085107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:27.085107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:27.085107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:27.085107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:27.085107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:27.085107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:27.085107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:27.085107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:27.085107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:27.085107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:27.085107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:27.085107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:27.085107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:27.086399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:27.086399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:27.086399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:42:27.086399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:42:27.086399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:42:27.086399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:42:27.086399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:42:27.086399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:42:27.086399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 10 23:42:27.086399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:27.087165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1493d08-3a8b-40a3-a24b-5f623eef07ca None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:27.087255 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 283/551] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:42:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:27.091908 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea8a22-0d0e-405f-9a38-504813c57ab4 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:27.094098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 269/552] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Sep 10 23:42:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:27.100434 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1493d08-3a8b-40a3-a24b-5f623eef07ca None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:27.223031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:27.241414 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea8a22-0d0e-405f-9a38-504813c57ab4 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:27.323768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1493d08-3a8b-40a3-a24b-5f623eef07ca None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:27.325891 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc9ad4-e74e-49be-9f89-ec3ccd8715eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:27.327879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc9ad4-e74e-49be-9f89-ec3ccd8715eb None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:27.328157 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc9ad4-e74e-49be-9f89-ec3ccd8715eb None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:27.328425 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc9ad4-e74e-49be-9f89-ec3ccd8715eb None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:27.329160 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc9ad4-e74e-49be-9f89-ec3ccd8715eb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:27.329463 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc9ad4-e74e-49be-9f89-ec3ccd8715eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:27.330933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc9ad4-e74e-49be-9f89-ec3ccd8715eb None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:27.373424 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 284/553] 10.222.0.17 () {56 vars in 1135 bytes} [Sat Sep 10 23:42:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:27.373736 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:27.378277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc9ad4-e74e-49be-9f89-ec3ccd8715eb None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:27.404479 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbaed76-bad8-45af-9dc4-8d2a601da4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:27.405157 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbaed76-bad8-45af-9dc4-8d2a601da4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:27.405433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbaed76-bad8-45af-9dc4-8d2a601da4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:27.405703 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbaed76-bad8-45af-9dc4-8d2a601da4d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:27.422523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbaed76-bad8-45af-9dc4-8d2a601da4d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:27.424479 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbaed76-bad8-45af-9dc4-8d2a601da4d2 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:27.444428 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-dfbaed76-bad8-45af-9dc4-8d2a601da4d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:27.444428 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:27.444428 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:27.444428 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:27.444428 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:27.444428 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:27.444428 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:27.444428 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:27.444428 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:27.444428 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:27.444428 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:27.444428 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:27.444428 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:27.444428 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:27.444428 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:27.444428 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:27.445410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:27.445410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:27.445410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:27.445410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:27.445410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:27.445410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:27.445410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:27.445410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:27.445410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:27.445410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:27.445410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:27.445410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:27.445410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:27.445410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:27.445410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:27.445410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:27.445410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:27.446257 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:27.446257 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:27.446257 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:27.447104 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 270/554] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:27.448057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a860015-8666-4e99-9550-7630ef134c9f None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:27.448340 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a860015-8666-4e99-9550-7630ef134c9f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:27.448611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a860015-8666-4e99-9550-7630ef134c9f None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:27.451734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 271/555] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:27.452539 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a860015-8666-4e99-9550-7630ef134c9f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:27.513192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc9ad4-e74e-49be-9f89-ec3ccd8715eb None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:27.556546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62073e4-89e3-4795-838f-8cecee802c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:27.557482 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62073e4-89e3-4795-838f-8cecee802c1f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:27.557814 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62073e4-89e3-4795-838f-8cecee802c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:27.558151 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62073e4-89e3-4795-838f-8cecee802c1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:27.578141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:27.578421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a860015-8666-4e99-9550-7630ef134c9f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:27.617091 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a41dfe-9ea3-4986-8594-b9fa70396e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:27.617691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a41dfe-9ea3-4986-8594-b9fa70396e91 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:27.618013 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a41dfe-9ea3-4986-8594-b9fa70396e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:27.618391 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a41dfe-9ea3-4986-8594-b9fa70396e91 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:27.619708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a41dfe-9ea3-4986-8594-b9fa70396e91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:27.620036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a41dfe-9ea3-4986-8594-b9fa70396e91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:27.621941 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a41dfe-9ea3-4986-8594-b9fa70396e91 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:27.637259 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 272/556] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:27.639078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e87f03-f059-4185-9930-b4d9665fa042 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:27.639363 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e87f03-f059-4185-9930-b4d9665fa042 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:27.639801 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e87f03-f059-4185-9930-b4d9665fa042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:27.648013 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62073e4-89e3-4795-838f-8cecee802c1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:27.649831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62073e4-89e3-4795-838f-8cecee802c1f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:27.659783 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d9e87f03-f059-4185-9930-b4d9665fa042 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:27.661142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d9e87f03-f059-4185-9930-b4d9665fa042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:27.689939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:27.742730 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 273/557] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:27.742730 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e87f03-f059-4185-9930-b4d9665fa042 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:27.793635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 285/558] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:42:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 10 23:42:27.799737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62073e4-89e3-4795-838f-8cecee802c1f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:27.877029 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:27.933024 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62073e4-89e3-4795-838f-8cecee802c1f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:27.936977 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e87f03-f059-4185-9930-b4d9665fa042 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:27.999875 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916abd2-b554-484b-8ac4-42b65c1de8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:28.000499 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916abd2-b554-484b-8ac4-42b65c1de8b2 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:28.000774 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916abd2-b554-484b-8ac4-42b65c1de8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:28.001041 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916abd2-b554-484b-8ac4-42b65c1de8b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:28.005625 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11948508-6959-45a5-940b-a2e40a0c973d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:28.006245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11948508-6959-45a5-940b-a2e40a0c973d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:28.006521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11948508-6959-45a5-940b-a2e40a0c973d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:28.006910 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11948508-6959-45a5-940b-a2e40a0c973d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:28.015090 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916abd2-b554-484b-8ac4-42b65c1de8b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:28.016587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916abd2-b554-484b-8ac4-42b65c1de8b2 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:28.024379 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11948508-6959-45a5-940b-a2e40a0c973d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:28.025849 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11948508-6959-45a5-940b-a2e40a0c973d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:28.032389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-7916abd2-b554-484b-8ac4-42b65c1de8b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:28.032389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:28.032389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:28.032389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:28.032389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:28.032389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:28.032389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:28.032389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:28.032389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:28.032389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:28.032389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:28.032389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:28.032389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:28.032389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:28.032389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:28.032389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:28.033301 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:28.033301 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:28.033301 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:28.033301 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:28.033301 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:28.033301 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:28.033301 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:28.033301 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:28.033301 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:28.033301 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:28.033301 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:28.033301 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:28.033301 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:28.033301 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:28.033301 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:28.033301 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:28.033301 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:28.034150 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:28.034150 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:28.034150 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:28.034150 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 286/559] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:28.040755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916abd2-b554-484b-8ac4-42b65c1de8b2 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:28.052163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-11948508-6959-45a5-940b-a2e40a0c973d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 23:42:28.052163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:28.052163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:28.052163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:28.052163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:28.052163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:28.052163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:28.052163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:28.052163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:28.052163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:28.052163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:28.052163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:28.052163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:42:28.052163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:42:28.052163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:42:28.052163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:42:28.053199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:28.053199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:28.053199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:42:28.053199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:28.053199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:42:28.053199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:28.053199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:28.053199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:28.053199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:28.053199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:28.053199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:28.053199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:28.053199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:28.053199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:28.053199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:28.053199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:28.053199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:28.053199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:28.054121 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:28.054121 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:28.054121 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:42:28.054121 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:42:28.054121 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:42:28.054121 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:42:28.054121 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:42:28.054121 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:42:28.054121 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 23:42:28.054121 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:28.054121 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 274/560] 10.222.0.17 () {56 vars in 1093 bytes} [Sat Sep 10 23:42:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:28.067628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11948508-6959-45a5-940b-a2e40a0c973d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:28.135127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:28.162247 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:28.169175 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916abd2-b554-484b-8ac4-42b65c1de8b2 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:28.203256 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11948508-6959-45a5-940b-a2e40a0c973d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:28.210378 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b6ebd-fc77-482a-83fd-d6b5351ef0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:28.211397 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b6ebd-fc77-482a-83fd-d6b5351ef0a2 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:28.211870 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b6ebd-fc77-482a-83fd-d6b5351ef0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:28.212334 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b6ebd-fc77-482a-83fd-d6b5351ef0a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:28.213224 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b6ebd-fc77-482a-83fd-d6b5351ef0a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:28.213755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b6ebd-fc77-482a-83fd-d6b5351ef0a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:28.215556 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b6ebd-fc77-482a-83fd-d6b5351ef0a2 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:28.236324 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 287/561] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:42:28.237717 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b6ebd-fc77-482a-83fd-d6b5351ef0a2 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:28.249254 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7a677-e2ba-4256-b29a-165deebf7abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:28.250317 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7a677-e2ba-4256-b29a-165deebf7abe None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:28.250607 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7a677-e2ba-4256-b29a-165deebf7abe None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:28.250932 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7a677-e2ba-4256-b29a-165deebf7abe None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:28.251674 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b7a677-e2ba-4256-b29a-165deebf7abe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:28.251980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b7a677-e2ba-4256-b29a-165deebf7abe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:28.254146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b7a677-e2ba-4256-b29a-165deebf7abe None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:28.282602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 275/562] 10.222.0.17 () {56 vars in 1133 bytes} [Sat Sep 10 23:42:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:28.283824 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7a677-e2ba-4256-b29a-165deebf7abe None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:28.364589 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:28.368208 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b6ebd-fc77-482a-83fd-d6b5351ef0a2 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:28.390701 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7a677-e2ba-4256-b29a-165deebf7abe None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:28.406062 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e860d9ce-ca16-4dbb-a2f1-1dc78938ac22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:28.406702 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860d9ce-ca16-4dbb-a2f1-1dc78938ac22 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:28.406976 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860d9ce-ca16-4dbb-a2f1-1dc78938ac22 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:28.407241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860d9ce-ca16-4dbb-a2f1-1dc78938ac22 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:28.418864 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e860d9ce-ca16-4dbb-a2f1-1dc78938ac22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:28.423560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-e860d9ce-ca16-4dbb-a2f1-1dc78938ac22 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:28.423560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:28.423560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:28.423560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:28.423560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:28.423560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:28.423560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:28.423560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:28.423560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:28.423560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:28.423560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:28.423560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:28.423560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:28.423560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:28.423560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:28.423560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:28.424559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:28.424559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:28.424559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:28.424559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:28.424559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:28.424559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:28.424559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:28.424559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:28.424559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:28.424559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:28.424559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:28.424559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:28.424559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:28.424559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:28.424559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:28.424559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:28.425384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:28.425384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:28.425384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:28.425384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:28.425384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:28.425384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:28.425384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:28.425384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:28.425384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:28.425384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:28.425384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:28.425384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:28.425384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 288/563] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Sep 10 23:42:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:28.425384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e860d9ce-ca16-4dbb-a2f1-1dc78938ac22 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:28.431445 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ec6dc-1bb6-4565-99fb-5bafcc69ed1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:28.432337 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ec6dc-1bb6-4565-99fb-5bafcc69ed1d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:28.432616 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ec6dc-1bb6-4565-99fb-5bafcc69ed1d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:28.432887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ec6dc-1bb6-4565-99fb-5bafcc69ed1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:28.441052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ec6dc-1bb6-4565-99fb-5bafcc69ed1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:28.442531 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ec6dc-1bb6-4565-99fb-5bafcc69ed1d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:28.445351 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:28.453974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-6b8ec6dc-1bb6-4565-99fb-5bafcc69ed1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:28.453974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:28.453974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:28.453974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:28.453974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:28.453974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:28.453974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:28.453974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:28.453974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:28.453974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:28.453974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:28.453974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:28.453974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:42:28.453974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:42:28.453974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:42:28.460926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:42:28.460926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:28.460926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:28.460926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:28.460926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:28.460926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:28.460926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:28.460926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:28.460926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:28.460926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:28.460926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:28.460926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:28.460926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:28.460926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:28.460926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:28.460926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:28.460926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:28.462392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:42:28.462392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:42:28.462392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:42:28.462392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:42:28.462392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:42:28.462392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:42:28.462392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:28.462392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:28.462392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 276/564] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:28.462392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f116988-c812-486f-b8e9-0517bdc67bed None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:28.462392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f116988-c812-486f-b8e9-0517bdc67bed None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:28.463050 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f116988-c812-486f-b8e9-0517bdc67bed None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:28.463050 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 277/565] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:28.463050 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f116988-c812-486f-b8e9-0517bdc67bed None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:28.545876 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e860d9ce-ca16-4dbb-a2f1-1dc78938ac22 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:28.569768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f116988-c812-486f-b8e9-0517bdc67bed None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:28.582908 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe101df-ba21-43d1-b17d-e0fbe9bd25ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:28.583572 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe101df-ba21-43d1-b17d-e0fbe9bd25ab None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:28.583799 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe101df-ba21-43d1-b17d-e0fbe9bd25ab None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:28.584118 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe101df-ba21-43d1-b17d-e0fbe9bd25ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:28.595770 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe101df-ba21-43d1-b17d-e0fbe9bd25ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:28.597549 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe101df-ba21-43d1-b17d-e0fbe9bd25ab None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:28.607633 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c962e3d-cef7-4a72-9643-f1579e8b1dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:28.608230 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c962e3d-cef7-4a72-9643-f1579e8b1dbe None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:28.608506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c962e3d-cef7-4a72-9643-f1579e8b1dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:28.608776 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c962e3d-cef7-4a72-9643-f1579e8b1dbe None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:28.609433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c962e3d-cef7-4a72-9643-f1579e8b1dbe None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:28.609733 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c962e3d-cef7-4a72-9643-f1579e8b1dbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:28.611123 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c962e3d-cef7-4a72-9643-f1579e8b1dbe None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:28.613486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-8fe101df-ba21-43d1-b17d-e0fbe9bd25ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:28.613486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:28.613486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:28.613486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:28.613486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:28.613486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:28.613486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:28.613486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:28.613486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:28.613486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:28.613486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:28.613486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:28.613486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:28.613486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:28.613486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:28.613486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:28.614406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:28.614406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:28.614406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:28.614406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:28.614406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:28.614406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:28.614406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:28.614406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:28.614406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:28.614406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:28.614406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:28.614406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:28.614406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:28.614406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:28.614406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:28.614406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:28.614406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:28.615854 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:28.615854 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:28.615854 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:28.616418 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:28.618856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 289/566] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:28.620402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2b8a7-ce44-4388-93a5-2b14832e5ea1 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:28.620686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2b8a7-ce44-4388-93a5-2b14832e5ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:28.620961 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2b8a7-ce44-4388-93a5-2b14832e5ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:28.626773 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 278/567] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Sep 10 23:42:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:28.627640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c962e3d-cef7-4a72-9643-f1579e8b1dbe None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:28.639904 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-e1d2b8a7-ce44-4388-93a5-2b14832e5ea1 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:28.641293 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e1d2b8a7-ce44-4388-93a5-2b14832e5ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:28.706894 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:28.712795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 290/568] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:28.713899 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2b8a7-ce44-4388-93a5-2b14832e5ea1 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:28.750597 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c962e3d-cef7-4a72-9643-f1579e8b1dbe None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:28.794276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4ac63-5cd6-464d-9341-628e63d64342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:28.795737 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4ac63-5cd6-464d-9341-628e63d64342 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:28.796167 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4ac63-5cd6-464d-9341-628e63d64342 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:28.796590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4ac63-5cd6-464d-9341-628e63d64342 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:28.797439 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4ac63-5cd6-464d-9341-628e63d64342 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:28.797886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4ac63-5cd6-464d-9341-628e63d64342 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:28.800137 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4ac63-5cd6-464d-9341-628e63d64342 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:28.817299 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 279/569] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:42:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:28.818405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4ac63-5cd6-464d-9341-628e63d64342 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:28.846715 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2b8a7-ce44-4388-93a5-2b14832e5ea1 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:28.881004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724bdd2-4c46-464e-9d46-81bb7ed5692b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:28.881612 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724bdd2-4c46-464e-9d46-81bb7ed5692b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:28.881891 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724bdd2-4c46-464e-9d46-81bb7ed5692b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:28.882164 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724bdd2-4c46-464e-9d46-81bb7ed5692b None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:28.882832 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724bdd2-4c46-464e-9d46-81bb7ed5692b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:28.883130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724bdd2-4c46-464e-9d46-81bb7ed5692b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:28.884510 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724bdd2-4c46-464e-9d46-81bb7ed5692b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:28.898056 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:28.906673 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 291/570] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:28.906673 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724bdd2-4c46-464e-9d46-81bb7ed5692b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:28.948107 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4ac63-5cd6-464d-9341-628e63d64342 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:28.965910 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:28.985600 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965fa49-ab86-434b-a6b5-451a30eb3150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:28.986224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965fa49-ab86-434b-a6b5-451a30eb3150 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:28.986495 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965fa49-ab86-434b-a6b5-451a30eb3150 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:28.986793 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965fa49-ab86-434b-a6b5-451a30eb3150 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:28.997805 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965fa49-ab86-434b-a6b5-451a30eb3150 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:28.999770 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965fa49-ab86-434b-a6b5-451a30eb3150 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:29.010565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724bdd2-4c46-464e-9d46-81bb7ed5692b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:29.014676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-8965fa49-ab86-434b-a6b5-451a30eb3150 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:29.014676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:29.014676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:29.014676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:29.014676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:29.014676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:29.014676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:29.014676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:29.014676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:29.014676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:29.014676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:29.014676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:29.014676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:29.014676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:29.014676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:29.014676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:29.015768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:29.015768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:29.015768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:29.015768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:29.015768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:29.015768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:29.015768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:29.015768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:29.015768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:29.015768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:29.015768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:29.015768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:29.015768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:29.015768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:29.015768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:29.015768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:29.015768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:29.016657 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:29.016657 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:29.016657 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:29.016657 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 280/571] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:29.016961 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97822d9-74e7-4928-9097-1e55bb2f6e9d None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:29.017185 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97822d9-74e7-4928-9097-1e55bb2f6e9d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:29.017429 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97822d9-74e7-4928-9097-1e55bb2f6e9d None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:29.018642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 281/572] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:29.019345 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97822d9-74e7-4928-9097-1e55bb2f6e9d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:29.056879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e1ca7-964a-4ef6-8b00-1af56189a39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:29.057729 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e1ca7-964a-4ef6-8b00-1af56189a39a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:29.058009 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e1ca7-964a-4ef6-8b00-1af56189a39a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:29.058276 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e1ca7-964a-4ef6-8b00-1af56189a39a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:29.059512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e1ca7-964a-4ef6-8b00-1af56189a39a None admin] RBAC: Adding query filter params (role.id=8ff1727b65104e5fb97333d477196d46, scope.project.id=947277c4458645899287e6818947c076, user.id=564e65585e9543e286b6041af870cf41) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:29.059816 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e1ca7-964a-4ef6-8b00-1af56189a39a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:29.061264 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e1ca7-964a-4ef6-8b00-1af56189a39a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:29.066437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 292/573] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:42:28 2022] GET /identity/v3/role_assignments?user.id=564e65585e9543e286b6041af870cf41&scope.project.id=947277c4458645899287e6818947c076&role.id=8ff1727b65104e5fb97333d477196d46 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:29.067237 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e1ca7-964a-4ef6-8b00-1af56189a39a None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:29.140463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97822d9-74e7-4928-9097-1e55bb2f6e9d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:29.155927 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:29.177983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374724c-ea1b-4dd7-851b-b80858caf436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:29.178611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374724c-ea1b-4dd7-851b-b80858caf436 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:29.178933 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374724c-ea1b-4dd7-851b-b80858caf436 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:29.179202 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374724c-ea1b-4dd7-851b-b80858caf436 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:29.187734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e1ca7-964a-4ef6-8b00-1af56189a39a None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:29.188710 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b374724c-ea1b-4dd7-851b-b80858caf436 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:29.190648 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b374724c-ea1b-4dd7-851b-b80858caf436 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:29.204072 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-b374724c-ea1b-4dd7-851b-b80858caf436 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:29.204072 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:29.204072 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:29.204072 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:29.204072 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:29.204072 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:29.204072 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:29.204072 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:29.204072 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:29.204072 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:29.204072 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:29.204072 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:29.204072 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:42:29.204072 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:42:29.204072 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:42:29.205098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:42:29.205098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:29.205098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:29.205098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:29.205098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:29.205098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:29.205098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:29.205098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:29.205098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:29.205098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:29.205098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:29.205098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:29.205098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:29.205098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:29.205098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:29.205098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:29.205098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:29.205999 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:42:29.205999 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:42:29.205999 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:42:29.205999 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:42:29.205999 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:42:29.205999 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:42:29.205999 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:29.205999 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:29.205999 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 282/574] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:29.207396 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374724c-ea1b-4dd7-851b-b80858caf436 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:29.220925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:29.234976 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0d575-e4f4-4b39-8f68-0082cd6af747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:29.235628 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0d575-e4f4-4b39-8f68-0082cd6af747 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:29.235897 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0d575-e4f4-4b39-8f68-0082cd6af747 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:29.236168 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0d575-e4f4-4b39-8f68-0082cd6af747 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:29.249004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0d575-e4f4-4b39-8f68-0082cd6af747 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:29.253801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-c5f0d575-e4f4-4b39-8f68-0082cd6af747 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:29.253801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:29.253801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:29.253801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:29.253801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:29.253801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:29.253801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:29.253801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:29.253801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:29.253801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:29.253801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:29.253801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:29.253801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:29.253801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:29.253801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:29.253801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:29.254879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:29.254879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:29.254879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:29.254879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:29.254879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:29.254879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:29.254879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:29.254879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:29.254879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:29.254879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:29.254879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:29.254879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:29.254879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:29.254879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:29.254879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:29.254879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:29.255735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:29.255735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:29.255735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:29.255735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:29.255735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:29.255735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:29.255735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:29.255735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:29.255735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:29.255735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:29.255735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:29.255735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:29.255735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 293/575] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Sep 10 23:42:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:29.257035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4bd72-d122-4c68-b97b-21ced36e53b0 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:29.257035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4bd72-d122-4c68-b97b-21ced36e53b0 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:29.257200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4bd72-d122-4c68-b97b-21ced36e53b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:29.276334 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-dcb4bd72-d122-4c68-b97b-21ced36e53b0 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:29.278121 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-dcb4bd72-d122-4c68-b97b-21ced36e53b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:29.323540 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374724c-ea1b-4dd7-851b-b80858caf436 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:29.366685 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 294/576] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:29.366685 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb4bd72-d122-4c68-b97b-21ced36e53b0 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:29.371510 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa995f33-fb82-4f63-bd03-97205842a087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:29.372759 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa995f33-fb82-4f63-bd03-97205842a087 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:29.373100 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa995f33-fb82-4f63-bd03-97205842a087 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:29.373442 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa995f33-fb82-4f63-bd03-97205842a087 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:29.374635 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa995f33-fb82-4f63-bd03-97205842a087 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:29.375025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa995f33-fb82-4f63-bd03-97205842a087 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:29.376938 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa995f33-fb82-4f63-bd03-97205842a087 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:29.392692 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 283/577] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:29.393359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa995f33-fb82-4f63-bd03-97205842a087 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:29.435719 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:29.479462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb4bd72-d122-4c68-b97b-21ced36e53b0 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:29.497393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:29.517618 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa995f33-fb82-4f63-bd03-97205842a087 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:29.521293 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612e370-db60-4b08-92b5-d0a91b7d5092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:29.521902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612e370-db60-4b08-92b5-d0a91b7d5092 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:29.522179 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612e370-db60-4b08-92b5-d0a91b7d5092 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:29.522446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612e370-db60-4b08-92b5-d0a91b7d5092 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:29.523189 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a612e370-db60-4b08-92b5-d0a91b7d5092 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:29.523489 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a612e370-db60-4b08-92b5-d0a91b7d5092 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:29.525005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a612e370-db60-4b08-92b5-d0a91b7d5092 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:29.542910 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 295/578] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:42:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:29.544380 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612e370-db60-4b08-92b5-d0a91b7d5092 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:29.560024 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0532f-29f8-4111-b06a-a5704fc60f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:29.560739 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0532f-29f8-4111-b06a-a5704fc60f8b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:29.561126 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0532f-29f8-4111-b06a-a5704fc60f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:29.561444 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0532f-29f8-4111-b06a-a5704fc60f8b None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:29.562177 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0532f-29f8-4111-b06a-a5704fc60f8b None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:29.562546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0532f-29f8-4111-b06a-a5704fc60f8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:29.564054 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0532f-29f8-4111-b06a-a5704fc60f8b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:29.572659 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 284/579] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Sep 10 23:42:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:29.573591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0532f-29f8-4111-b06a-a5704fc60f8b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:29.669176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612e370-db60-4b08-92b5-d0a91b7d5092 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:29.679799 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:29.692938 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0532f-29f8-4111-b06a-a5704fc60f8b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:29.708304 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a2f99-dac4-42c3-b942-dc6bc335aa54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:29.709020 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a2f99-dac4-42c3-b942-dc6bc335aa54 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:29.709305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a2f99-dac4-42c3-b942-dc6bc335aa54 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:29.709594 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a2f99-dac4-42c3-b942-dc6bc335aa54 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:29.717979 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a2f99-dac4-42c3-b942-dc6bc335aa54 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:29.719448 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a2f99-dac4-42c3-b942-dc6bc335aa54 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:29.722735 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c54d15-702d-4690-9f35-926c80ddd796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:29.723384 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c54d15-702d-4690-9f35-926c80ddd796 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:29.723660 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c54d15-702d-4690-9f35-926c80ddd796 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:29.723985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c54d15-702d-4690-9f35-926c80ddd796 None admin] PATH_INFO: `/v3/projects/947277c4458645899287e6818947c076/users/636bfafb693044d283530063047fc0c2/roles/8ff1727b65104e5fb97333d477196d46` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:29.731070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-120a2f99-dac4-42c3-b942-dc6bc335aa54 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 23:42:29.731070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:29.731070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:29.731070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:29.731070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:29.731070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:29.731070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:29.731070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:29.731070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:29.731070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:29.731070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:29.731070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:29.731070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:42:29.731070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:42:29.731070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:42:29.731070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:42:29.732137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:29.732137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:29.732137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:42:29.732137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:29.732137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:42:29.732137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:29.732137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:29.732137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:29.732137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:29.732137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:29.732137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:29.732137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:29.732137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:29.732137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:29.732137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:29.732137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:29.732137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:29.732137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:29.737903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:29.737903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:29.737903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:42:29.737903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:42:29.737903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:42:29.737903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:42:29.737903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:42:29.737903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:42:29.737903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 23:42:29.737903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:29.737903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 296/580] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:42:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:29.737903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a2f99-dac4-42c3-b942-dc6bc335aa54 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:29.745901 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c54d15-702d-4690-9f35-926c80ddd796 None admin] RBAC: Authorizing `identity:create_grant(project_id=947277c4458645899287e6818947c076, user_id=636bfafb693044d283530063047fc0c2, role_id=8ff1727b65104e5fb97333d477196d46)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:29.748044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c54d15-702d-4690-9f35-926c80ddd796 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:29.769075 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:29.826698 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 285/581] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Sep 10 23:42:29 2022] PUT /identity/v3/projects/947277c4458645899287e6818947c076/users/636bfafb693044d283530063047fc0c2/roles/8ff1727b65104e5fb97333d477196d46 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:42:29.826698 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c54d15-702d-4690-9f35-926c80ddd796 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:29.848668 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a2f99-dac4-42c3-b942-dc6bc335aa54 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:29.890503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b4b42-82aa-4cc3-818f-50da335ab617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:29.891174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b4b42-82aa-4cc3-818f-50da335ab617 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:29.891464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b4b42-82aa-4cc3-818f-50da335ab617 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:29.892093 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b4b42-82aa-4cc3-818f-50da335ab617 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:29.906456 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b4b42-82aa-4cc3-818f-50da335ab617 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:29.908104 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b4b42-82aa-4cc3-818f-50da335ab617 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:29.925440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-839b4b42-82aa-4cc3-818f-50da335ab617 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:29.925440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:29.925440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:29.925440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:29.925440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:29.925440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:29.925440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:29.925440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:29.925440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:29.925440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:29.925440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:29.925440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:29.925440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:29.925440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:29.925440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:29.925440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:29.926631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:29.926631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:29.926631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:29.926631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:29.926631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:29.926631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:29.926631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:29.926631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:29.926631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:29.926631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:29.926631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:29.926631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:29.926631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:29.926631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:29.926631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:29.926631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:29.926631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:29.927738 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:29.927738 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:29.927738 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:29.928350 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 297/582] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:29.929412 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b4b42-82aa-4cc3-818f-50da335ab617 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:29.951761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c54d15-702d-4690-9f35-926c80ddd796 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:29.991435 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645e816-83b0-4ae6-9df5-0ad262cb396a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:29.992171 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645e816-83b0-4ae6-9df5-0ad262cb396a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:29.992562 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645e816-83b0-4ae6-9df5-0ad262cb396a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:29.992904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645e816-83b0-4ae6-9df5-0ad262cb396a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:30.006092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645e816-83b0-4ae6-9df5-0ad262cb396a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:30.011907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-1645e816-83b0-4ae6-9df5-0ad262cb396a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:30.011907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:30.011907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:30.011907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:30.011907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:30.011907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:30.011907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:30.011907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:30.011907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:30.011907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:30.011907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:30.011907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:30.011907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:30.011907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:30.011907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:30.011907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:30.013521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:30.013521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:30.013521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:30.013521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:30.013521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:30.013521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:30.013521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:30.013521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:30.013521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:30.013521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:30.013521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:30.013521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:30.013521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:30.013521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:30.013521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:30.013521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:30.014546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:30.014546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:30.014546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:30.014546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:30.014546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:30.014546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:30.014546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:30.014546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:30.014546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:30.014546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:30.014546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:30.014546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:30.020635 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 286/583] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Sep 10 23:42:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:30.020635 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645e816-83b0-4ae6-9df5-0ad262cb396a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:30.038242 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:30.049993 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:30.069597 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b4b42-82aa-4cc3-818f-50da335ab617 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:30.114606 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831e808-08e1-4342-b38f-c4538e39b5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:30.115580 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831e808-08e1-4342-b38f-c4538e39b5c7 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:30.116136 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831e808-08e1-4342-b38f-c4538e39b5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:30.116408 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831e808-08e1-4342-b38f-c4538e39b5c7 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:30.117200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831e808-08e1-4342-b38f-c4538e39b5c7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:30.117662 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831e808-08e1-4342-b38f-c4538e39b5c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:30.119832 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831e808-08e1-4342-b38f-c4538e39b5c7 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:30.138305 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645e816-83b0-4ae6-9df5-0ad262cb396a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:30.143142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 298/584] 10.222.0.17 () {56 vars in 1137 bytes} [Sat Sep 10 23:42:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:30.143981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831e808-08e1-4342-b38f-c4538e39b5c7 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:30.176306 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d51fbf-f516-4e69-b6e8-051e2c958af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:30.176905 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d51fbf-f516-4e69-b6e8-051e2c958af4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:30.177182 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d51fbf-f516-4e69-b6e8-051e2c958af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:30.177448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d51fbf-f516-4e69-b6e8-051e2c958af4 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:30.178183 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d51fbf-f516-4e69-b6e8-051e2c958af4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:30.178508 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d51fbf-f516-4e69-b6e8-051e2c958af4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:30.179992 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d51fbf-f516-4e69-b6e8-051e2c958af4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:30.193236 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 287/585] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:30.198632 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d51fbf-f516-4e69-b6e8-051e2c958af4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:30.272555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831e808-08e1-4342-b38f-c4538e39b5c7 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:30.279108 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:30.316043 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0e58c-0935-4ea8-968f-c3b9f9c0b808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:30.317946 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0e58c-0935-4ea8-968f-c3b9f9c0b808 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:30.318944 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0e58c-0935-4ea8-968f-c3b9f9c0b808 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:30.319406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0e58c-0935-4ea8-968f-c3b9f9c0b808 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:30.320291 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f0e58c-0935-4ea8-968f-c3b9f9c0b808 None admin] RBAC: Adding query filter params (name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:30.320597 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d51fbf-f516-4e69-b6e8-051e2c958af4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:30.320973 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f0e58c-0935-4ea8-968f-c3b9f9c0b808 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:30.322820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f0e58c-0935-4ea8-968f-c3b9f9c0b808 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:30.332439 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 299/586] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Sep 10 23:42:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:30.333485 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0e58c-0935-4ea8-968f-c3b9f9c0b808 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:30.338310 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:30.367356 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b811206-8d1e-4130-b642-d8a442544955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:30.368431 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b811206-8d1e-4130-b642-d8a442544955 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:30.368807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b811206-8d1e-4130-b642-d8a442544955 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:30.369324 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b811206-8d1e-4130-b642-d8a442544955 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:30.385223 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b811206-8d1e-4130-b642-d8a442544955 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:30.387415 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b811206-8d1e-4130-b642-d8a442544955 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:30.402958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-0b811206-8d1e-4130-b642-d8a442544955 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:30.402958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:30.402958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:30.402958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:30.402958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:30.402958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:30.402958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:30.402958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:30.402958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:30.402958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:30.402958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:30.402958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:30.402958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:30.402958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:30.402958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:30.402958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:30.403939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:30.403939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:30.403939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:30.403939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:30.403939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:30.403939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:30.403939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:30.403939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:30.403939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:30.403939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:30.403939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:30.403939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:30.403939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:30.403939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:30.403939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:30.403939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:30.403939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:30.404804 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:30.404804 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:30.404804 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:30.404804 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 288/587] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:30.405436 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b811206-8d1e-4130-b642-d8a442544955 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:30.465628 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0e58c-0935-4ea8-968f-c3b9f9c0b808 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:30.507069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d83780-635e-4ec3-b96b-78e63598351c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:30.508238 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d83780-635e-4ec3-b96b-78e63598351c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:30.508996 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d83780-635e-4ec3-b96b-78e63598351c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:30.509715 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d83780-635e-4ec3-b96b-78e63598351c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:30.515524 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b811206-8d1e-4130-b642-d8a442544955 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:30.520717 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d83780-635e-4ec3-b96b-78e63598351c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:30.522423 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d83780-635e-4ec3-b96b-78e63598351c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:30.534842 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:30.538587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-69d83780-635e-4ec3-b96b-78e63598351c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 10 23:42:30.538587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:30.538587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:30.538587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:30.538587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:30.538587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:30.538587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:30.538587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:30.538587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:30.538587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:30.538587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:30.538587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:30.538587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:42:30.538587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:42:30.538587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:42:30.538587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:42:30.539614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:30.539614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:30.539614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:42:30.539614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:30.539614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:42:30.539614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:30.539614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:30.539614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:30.539614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:30.539614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:30.539614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:30.539614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:30.539614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:30.539614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:30.539614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:30.539614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:30.539614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:30.539614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:30.540517 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:30.540517 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:30.540517 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:42:30.540517 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:42:30.540517 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:42:30.540517 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:42:30.540517 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:42:30.540517 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:42:30.540517 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 10 23:42:30.540517 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:30.548892 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 300/588] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:42:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:30.548892 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d83780-635e-4ec3-b96b-78e63598351c None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:30.561810 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2be143-e6d0-4a5b-85d6-3ab02ca6391c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:30.562425 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2be143-e6d0-4a5b-85d6-3ab02ca6391c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:30.562787 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2be143-e6d0-4a5b-85d6-3ab02ca6391c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:30.563059 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2be143-e6d0-4a5b-85d6-3ab02ca6391c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:30.577421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2be143-e6d0-4a5b-85d6-3ab02ca6391c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:30.580265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2be143-e6d0-4a5b-85d6-3ab02ca6391c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:30.597011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-da2be143-e6d0-4a5b-85d6-3ab02ca6391c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:30.597011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:30.597011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:30.597011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:30.597011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:30.597011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:30.597011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:30.597011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:30.597011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:30.597011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:30.597011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:30.597011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:30.597011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:30.597011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:30.597011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:30.597011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:30.598078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:30.598078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:30.598078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:30.598078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:30.598078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:30.598078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:30.598078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:30.598078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:30.598078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:30.598078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:30.598078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:30.598078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:30.598078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:30.598078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:30.598078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:30.598078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:30.598078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:30.598961 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:30.598961 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:30.598961 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:30.601056 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 289/589] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:30.602218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2be143-e6d0-4a5b-85d6-3ab02ca6391c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:30.630054 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:30.669705 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d83780-635e-4ec3-b96b-78e63598351c None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:30.705169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3adf07e-8f84-4931-8557-1233f485aa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:30.705767 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adf07e-8f84-4931-8557-1233f485aa42 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:30.706056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adf07e-8f84-4931-8557-1233f485aa42 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:30.706502 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adf07e-8f84-4931-8557-1233f485aa42 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:30.706960 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3adf07e-8f84-4931-8557-1233f485aa42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:30.707254 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3adf07e-8f84-4931-8557-1233f485aa42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:30.708601 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3adf07e-8f84-4931-8557-1233f485aa42 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:30.716589 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2be143-e6d0-4a5b-85d6-3ab02ca6391c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:30.721021 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 301/590] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:30.721849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3adf07e-8f84-4931-8557-1233f485aa42 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:30.755532 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bfe83c-6993-49ed-8c69-9f940625cc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:30.756620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bfe83c-6993-49ed-8c69-9f940625cc3a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:30.757226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bfe83c-6993-49ed-8c69-9f940625cc3a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:30.757686 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bfe83c-6993-49ed-8c69-9f940625cc3a None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:30.758637 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bfe83c-6993-49ed-8c69-9f940625cc3a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:30.759155 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bfe83c-6993-49ed-8c69-9f940625cc3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:30.760838 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bfe83c-6993-49ed-8c69-9f940625cc3a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:30.785196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 290/591] 10.222.0.17 () {56 vars in 1137 bytes} [Sat Sep 10 23:42:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:30.786432 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bfe83c-6993-49ed-8c69-9f940625cc3a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:30.816211 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:30.824566 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3adf07e-8f84-4931-8557-1233f485aa42 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:30.850620 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:30.861974 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363872b-1600-4aa8-80ae-a1e870046ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:30.863742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363872b-1600-4aa8-80ae-a1e870046ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:30.863742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363872b-1600-4aa8-80ae-a1e870046ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:30.863742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363872b-1600-4aa8-80ae-a1e870046ed2 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:30.863742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363872b-1600-4aa8-80ae-a1e870046ed2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:30.864094 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363872b-1600-4aa8-80ae-a1e870046ed2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:30.865364 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363872b-1600-4aa8-80ae-a1e870046ed2 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:30.877997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 302/592] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:30.878627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363872b-1600-4aa8-80ae-a1e870046ed2 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:30.901733 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bfe83c-6993-49ed-8c69-9f940625cc3a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:30.941148 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8907a7b9-0f29-4f87-9f97-a95736e7ece1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:30.941936 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907a7b9-0f29-4f87-9f97-a95736e7ece1 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:30.942352 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907a7b9-0f29-4f87-9f97-a95736e7ece1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:30.942834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907a7b9-0f29-4f87-9f97-a95736e7ece1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:30.950310 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8907a7b9-0f29-4f87-9f97-a95736e7ece1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:30.952045 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8907a7b9-0f29-4f87-9f97-a95736e7ece1 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:30.964136 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-8907a7b9-0f29-4f87-9f97-a95736e7ece1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:30.964136 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:30.964136 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:30.964136 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:30.964136 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:30.964136 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:30.964136 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:30.964136 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:30.964136 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:30.964136 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:30.964136 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:30.964136 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:30.964136 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:42:30.964136 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:42:30.964136 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:42:30.965611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:42:30.965611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:30.965611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:30.965611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:30.965611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:30.965611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:30.965611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:30.965611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:30.965611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:30.965611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:30.965611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:30.965611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:30.965611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:30.965611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:30.965611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:30.965611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:30.965611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:30.966708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:42:30.966708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:42:30.966708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:42:30.966708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:42:30.966708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:42:30.966708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:42:30.966708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:30.966708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:30.966708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 291/593] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:30.968241 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8907a7b9-0f29-4f87-9f97-a95736e7ece1 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:30.983859 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363872b-1600-4aa8-80ae-a1e870046ed2 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:31.018555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b27c97-8e8c-426d-a29a-712669904cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:31.019212 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b27c97-8e8c-426d-a29a-712669904cf9 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:31.019491 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b27c97-8e8c-426d-a29a-712669904cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:31.019761 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b27c97-8e8c-426d-a29a-712669904cf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:31.031460 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b27c97-8e8c-426d-a29a-712669904cf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:31.032897 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b27c97-8e8c-426d-a29a-712669904cf9 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:31.048165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-88b27c97-8e8c-426d-a29a-712669904cf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:31.048165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:31.048165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:31.048165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:31.048165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:31.048165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:31.048165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:31.048165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:31.048165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:31.048165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:31.048165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:31.048165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:31.048165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:31.048165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:31.048165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:31.048165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:31.050383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:31.050383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:31.050383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:31.050383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:31.050383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:31.050383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:31.050383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:31.050383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:31.050383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:31.050383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:31.050383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:31.050383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:31.050383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:31.050383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:31.050383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:31.050383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:31.050383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:31.052544 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:31.052544 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:31.052544 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:31.058684 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 303/594] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:31.058684 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b27c97-8e8c-426d-a29a-712669904cf9 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:31.059077 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:31.073562 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8907a7b9-0f29-4f87-9f97-a95736e7ece1 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:31.091359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:31.109683 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af61703-b366-496c-88bd-12bd69a0c9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:31.110301 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af61703-b366-496c-88bd-12bd69a0c9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:31.110574 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af61703-b366-496c-88bd-12bd69a0c9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:31.110867 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af61703-b366-496c-88bd-12bd69a0c9d5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:31.118437 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af61703-b366-496c-88bd-12bd69a0c9d5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:31.119884 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af61703-b366-496c-88bd-12bd69a0c9d5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:31.131861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-3af61703-b366-496c-88bd-12bd69a0c9d5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 10 23:42:31.131861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:31.131861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:31.131861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:31.131861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:31.131861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:31.131861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:31.131861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:31.131861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:31.131861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:31.131861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:31.131861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:31.131861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:42:31.131861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:42:31.131861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:42:31.131861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:42:31.133032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:31.133032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:31.133032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:42:31.133032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:31.133032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:42:31.133032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:31.133032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:31.133032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:31.133032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:31.133032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:31.133032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:31.133032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:31.133032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:31.133032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:31.133032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:31.133032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:31.133032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:31.133032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:31.134068 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:31.134068 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:31.134068 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:42:31.134068 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:42:31.134068 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:42:31.134068 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:42:31.134068 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:42:31.134068 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:42:31.134068 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 10 23:42:31.134068 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:31.134068 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 292/595] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Sep 10 23:42:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:31.134648 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af61703-b366-496c-88bd-12bd69a0c9d5 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:31.164711 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b27c97-8e8c-426d-a29a-712669904cf9 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:31.202956 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52aa1de-b322-4a33-a9bc-e8c6ee971ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:31.203927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52aa1de-b322-4a33-a9bc-e8c6ee971ced None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:31.204395 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52aa1de-b322-4a33-a9bc-e8c6ee971ced None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:31.204803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52aa1de-b322-4a33-a9bc-e8c6ee971ced None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:31.205660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52aa1de-b322-4a33-a9bc-e8c6ee971ced None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:31.206112 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52aa1de-b322-4a33-a9bc-e8c6ee971ced None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:31.207784 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52aa1de-b322-4a33-a9bc-e8c6ee971ced None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:31.222795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 304/596] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:42:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:31.224278 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52aa1de-b322-4a33-a9bc-e8c6ee971ced None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:31.242122 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af61703-b366-496c-88bd-12bd69a0c9d5 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:31.276845 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17278c4c-cc1f-4425-86f2-2adce21bb2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:31.278188 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17278c4c-cc1f-4425-86f2-2adce21bb2e6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:31.278466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17278c4c-cc1f-4425-86f2-2adce21bb2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:31.278764 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17278c4c-cc1f-4425-86f2-2adce21bb2e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:31.279431 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17278c4c-cc1f-4425-86f2-2adce21bb2e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:31.279687 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17278c4c-cc1f-4425-86f2-2adce21bb2e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:31.281125 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17278c4c-cc1f-4425-86f2-2adce21bb2e6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:31.297560 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 293/597] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:31.297560 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17278c4c-cc1f-4425-86f2-2adce21bb2e6 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:31.310711 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:31.336114 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:31.342306 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52aa1de-b322-4a33-a9bc-e8c6ee971ced None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:31.378267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e136667-e029-4dd6-99be-a90a917caa5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:31.378906 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e136667-e029-4dd6-99be-a90a917caa5e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:31.379180 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e136667-e029-4dd6-99be-a90a917caa5e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:31.379450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e136667-e029-4dd6-99be-a90a917caa5e None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:31.380168 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e136667-e029-4dd6-99be-a90a917caa5e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:31.380467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e136667-e029-4dd6-99be-a90a917caa5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:31.382127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e136667-e029-4dd6-99be-a90a917caa5e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:31.396251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17278c4c-cc1f-4425-86f2-2adce21bb2e6 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:31.405508 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 305/598] 10.222.0.17 () {56 vars in 1139 bytes} [Sat Sep 10 23:42:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:31.405508 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e136667-e029-4dd6-99be-a90a917caa5e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:31.433246 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d73a18-0309-487d-b4d5-b85408dbcd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:31.434069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d73a18-0309-487d-b4d5-b85408dbcd73 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:31.434342 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d73a18-0309-487d-b4d5-b85408dbcd73 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:31.434615 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d73a18-0309-487d-b4d5-b85408dbcd73 None admin] PATH_INFO: `/v3/projects/947277c4458645899287e6818947c076/users/c6fccf47e1ec4324983c5218ec3c96a4/roles/8ff1727b65104e5fb97333d477196d46` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:31.448872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d73a18-0309-487d-b4d5-b85408dbcd73 None admin] RBAC: Authorizing `identity:create_grant(project_id=947277c4458645899287e6818947c076, user_id=c6fccf47e1ec4324983c5218ec3c96a4, role_id=8ff1727b65104e5fb97333d477196d46)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:31.450909 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d73a18-0309-487d-b4d5-b85408dbcd73 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:31.499752 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e136667-e029-4dd6-99be-a90a917caa5e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:31.525717 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 294/599] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Sep 10 23:42:31 2022] PUT /identity/v3/projects/947277c4458645899287e6818947c076/users/c6fccf47e1ec4324983c5218ec3c96a4/roles/8ff1727b65104e5fb97333d477196d46 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:42:31.525717 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f7cc0-feed-499f-96a0-6769536b2950 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:31.525717 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f7cc0-feed-499f-96a0-6769536b2950 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:31.525717 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f7cc0-feed-499f-96a0-6769536b2950 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:31.525717 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 295/600] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:31.525717 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f7cc0-feed-499f-96a0-6769536b2950 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:31.551521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6dc4f-4620-4bac-b5c0-627d3cb8df1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:31.552554 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6dc4f-4620-4bac-b5c0-627d3cb8df1f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:31.552835 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6dc4f-4620-4bac-b5c0-627d3cb8df1f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:31.553104 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6dc4f-4620-4bac-b5c0-627d3cb8df1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:31.563348 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6dc4f-4620-4bac-b5c0-627d3cb8df1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:31.565735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6dc4f-4620-4bac-b5c0-627d3cb8df1f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:31.579724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-a7f6dc4f-4620-4bac-b5c0-627d3cb8df1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:31.579724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:31.579724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:31.579724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:31.579724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:31.579724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:31.579724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:31.579724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:31.579724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:31.579724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:31.579724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:31.579724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:31.579724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:42:31.579724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:42:31.579724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:42:31.580669 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:42:31.580669 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:31.580669 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:31.580669 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:31.580669 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:31.580669 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:31.580669 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:31.580669 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:31.580669 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:31.580669 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:31.580669 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:31.580669 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:31.580669 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:31.580669 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:31.580669 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:31.580669 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:31.580669 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:31.581573 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:42:31.581573 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:42:31.581573 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:42:31.581573 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:42:31.581573 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:42:31.581573 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:42:31.581573 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:31.581573 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:31.586391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 306/601] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:31.587894 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735cda3-b620-4cd2-b08f-66e7248c04de None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:31.588615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735cda3-b620-4cd2-b08f-66e7248c04de None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:31.588903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735cda3-b620-4cd2-b08f-66e7248c04de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:31.608552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:31.611956 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-5735cda3-b620-4cd2-b08f-66e7248c04de None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:31.613622 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5735cda3-b620-4cd2-b08f-66e7248c04de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:31.630789 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:31.671287 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f7cc0-feed-499f-96a0-6769536b2950 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:31.713718 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 307/602] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:31.715645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5735cda3-b620-4cd2-b08f-66e7248c04de None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:31.728905 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882bccd-d04f-4a31-be39-529890185e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:31.729606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882bccd-d04f-4a31-be39-529890185e67 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:31.730190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882bccd-d04f-4a31-be39-529890185e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:31.730341 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882bccd-d04f-4a31-be39-529890185e67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:31.742485 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f882bccd-d04f-4a31-be39-529890185e67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:31.744162 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f882bccd-d04f-4a31-be39-529890185e67 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:31.765380 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-f882bccd-d04f-4a31-be39-529890185e67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:31.765380 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:31.765380 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:31.765380 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:31.765380 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:31.765380 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:31.765380 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:31.765380 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:31.765380 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:31.765380 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:31.765380 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:31.765380 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:31.765380 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:31.765380 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:31.765380 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:31.765380 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:31.766373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:31.766373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:31.766373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:31.766373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:31.766373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:31.766373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:31.766373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:31.766373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:31.766373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:31.766373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:31.766373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:31.766373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:31.766373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:31.766373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:31.766373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:31.766373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:31.766373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:31.767294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:31.767294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:31.767294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:31.767294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 296/603] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:31.773759 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882bccd-d04f-4a31-be39-529890185e67 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:31.843139 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5735cda3-b620-4cd2-b08f-66e7248c04de None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:31.890290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e176e9-fb45-4f10-9a86-bdbe550e1c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:31.891001 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e176e9-fb45-4f10-9a86-bdbe550e1c28 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:31.891225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e176e9-fb45-4f10-9a86-bdbe550e1c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:31.891451 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e176e9-fb45-4f10-9a86-bdbe550e1c28 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:31.892256 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e176e9-fb45-4f10-9a86-bdbe550e1c28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:31.892565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e176e9-fb45-4f10-9a86-bdbe550e1c28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:31.894111 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e176e9-fb45-4f10-9a86-bdbe550e1c28 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:31.899323 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882bccd-d04f-4a31-be39-529890185e67 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:31.911013 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:31.921717 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 308/604] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:42:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:31.928058 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e176e9-fb45-4f10-9a86-bdbe550e1c28 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:31.936002 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:31.940457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014353a-7c6d-46c0-b566-5b8082bcea7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:31.941829 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014353a-7c6d-46c0-b566-5b8082bcea7f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:31.942290 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014353a-7c6d-46c0-b566-5b8082bcea7f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:31.942813 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014353a-7c6d-46c0-b566-5b8082bcea7f None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:31.943721 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014353a-7c6d-46c0-b566-5b8082bcea7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:31.944373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014353a-7c6d-46c0-b566-5b8082bcea7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:31.946284 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014353a-7c6d-46c0-b566-5b8082bcea7f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:31.960986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 297/605] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:42:31.968178 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014353a-7c6d-46c0-b566-5b8082bcea7f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:32.083749 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e176e9-fb45-4f10-9a86-bdbe550e1c28 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:32.118342 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014353a-7c6d-46c0-b566-5b8082bcea7f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:32.129099 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f7ba2-77c1-4db5-8828-cc66ac26e3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:32.129779 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f7ba2-77c1-4db5-8828-cc66ac26e3b6 None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:32.130056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f7ba2-77c1-4db5-8828-cc66ac26e3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:32.130438 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f7ba2-77c1-4db5-8828-cc66ac26e3b6 None admin] PATH_INFO: `/v3/projects/947277c4458645899287e6818947c076/users/2cb18199dc8b42589184042556e98983/roles/8ff1727b65104e5fb97333d477196d46` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:32.148958 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441f7ba2-77c1-4db5-8828-cc66ac26e3b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=947277c4458645899287e6818947c076, user_id=2cb18199dc8b42589184042556e98983, role_id=8ff1727b65104e5fb97333d477196d46)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:32.151266 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441f7ba2-77c1-4db5-8828-cc66ac26e3b6 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:32.155305 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa311520-873c-4f34-9686-1ea3d40f1c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:32.155947 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa311520-873c-4f34-9686-1ea3d40f1c4c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:32.156217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa311520-873c-4f34-9686-1ea3d40f1c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:32.156487 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa311520-873c-4f34-9686-1ea3d40f1c4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:32.163497 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa311520-873c-4f34-9686-1ea3d40f1c4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:32.164936 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa311520-873c-4f34-9686-1ea3d40f1c4c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:32.177971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-fa311520-873c-4f34-9686-1ea3d40f1c4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:32.177971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:32.177971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:32.177971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:32.177971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:32.177971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:32.177971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:32.177971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:32.177971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:32.177971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:32.177971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:32.177971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:32.177971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:42:32.177971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:42:32.177971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:42:32.179070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:42:32.179070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:32.179070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:32.179070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:32.179070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:32.179070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:32.179070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:32.179070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:32.179070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:32.179070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:32.179070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:32.179070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:32.179070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:32.179070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:32.179070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:32.179070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:32.179070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:32.185181 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:42:32.185181 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:42:32.185181 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:42:32.185181 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:42:32.185181 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:42:32.185181 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:42:32.185181 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:32.185181 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:32.185181 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 298/606] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:32.185181 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa311520-873c-4f34-9686-1ea3d40f1c4c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:32.211604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:32.242721 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 309/607] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Sep 10 23:42:31 2022] PUT /identity/v3/projects/947277c4458645899287e6818947c076/users/2cb18199dc8b42589184042556e98983/roles/8ff1727b65104e5fb97333d477196d46 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 23:42:32.242721 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f7ba2-77c1-4db5-8828-cc66ac26e3b6 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:32.295224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa311520-873c-4f34-9686-1ea3d40f1c4c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:32.316518 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:32.344383 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1a77c-6770-4328-bec9-025de725a712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:32.345410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1a77c-6770-4328-bec9-025de725a712 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:32.345691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1a77c-6770-4328-bec9-025de725a712 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:32.345963 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1a77c-6770-4328-bec9-025de725a712 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:32.360016 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1a77c-6770-4328-bec9-025de725a712 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:32.365130 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-7ce1a77c-6770-4328-bec9-025de725a712 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:32.365130 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:32.365130 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:32.365130 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:32.365130 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:32.365130 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:32.365130 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:32.365130 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:32.365130 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:32.365130 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:32.365130 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:32.365130 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:32.365130 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:32.365130 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:32.365130 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:32.365130 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:32.367903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:32.367903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:32.367903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:32.367903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:32.367903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:32.367903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:32.367903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:32.367903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:32.367903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:32.367903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:32.367903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:32.367903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:32.367903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:32.367903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:32.367903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:32.367903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:32.369049 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:32.369049 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:32.369049 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:32.369049 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:32.369049 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:32.369049 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:32.369049 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:32.369049 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:32.369049 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:32.369049 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:32.369049 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:32.369049 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:32.373898 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 299/608] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Sep 10 23:42:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:32.373898 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2693a3-f432-4e88-b629-ba8aa94d31aa None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:32.373898 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2693a3-f432-4e88-b629-ba8aa94d31aa None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:32.374170 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2693a3-f432-4e88-b629-ba8aa94d31aa None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:32.376028 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 300/609] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:32.376779 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2693a3-f432-4e88-b629-ba8aa94d31aa None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:32.377280 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f7ba2-77c1-4db5-8828-cc66ac26e3b6 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:32.423201 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99e46b-dbdb-4b9e-b59f-6473c7765670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:32.424107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99e46b-dbdb-4b9e-b59f-6473c7765670 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:32.424474 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99e46b-dbdb-4b9e-b59f-6473c7765670 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:32.424840 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99e46b-dbdb-4b9e-b59f-6473c7765670 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:32.425640 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99e46b-dbdb-4b9e-b59f-6473c7765670 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:32.426733 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99e46b-dbdb-4b9e-b59f-6473c7765670 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:32.428322 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99e46b-dbdb-4b9e-b59f-6473c7765670 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:32.443119 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 310/610] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:42:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:32.445149 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93eaa31-15b0-41d0-b069-5ba764807e58 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:32.445433 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93eaa31-15b0-41d0-b069-5ba764807e58 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:32.445708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93eaa31-15b0-41d0-b069-5ba764807e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:32.465621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-f93eaa31-15b0-41d0-b069-5ba764807e58 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:32.466841 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f93eaa31-15b0-41d0-b069-5ba764807e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:32.499492 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:32.516977 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2693a3-f432-4e88-b629-ba8aa94d31aa None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:32.538301 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 311/611] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:32.539164 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93eaa31-15b0-41d0-b069-5ba764807e58 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:32.569834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa6031-dd0d-46a6-b5ee-856ab8f19b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:32.570433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa6031-dd0d-46a6-b5ee-856ab8f19b60 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:32.570737 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa6031-dd0d-46a6-b5ee-856ab8f19b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:32.571009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa6031-dd0d-46a6-b5ee-856ab8f19b60 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:32.571731 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa6031-dd0d-46a6-b5ee-856ab8f19b60 None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:32.572036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa6031-dd0d-46a6-b5ee-856ab8f19b60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:32.573421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa6031-dd0d-46a6-b5ee-856ab8f19b60 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:32.590225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 301/612] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Sep 10 23:42:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:32.596002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa6031-dd0d-46a6-b5ee-856ab8f19b60 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:32.630985 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:32.690928 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93eaa31-15b0-41d0-b069-5ba764807e58 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:32.732023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03286d-217c-4469-aa87-059cbf8d91e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:32.733335 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03286d-217c-4469-aa87-059cbf8d91e7 None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:32.733764 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03286d-217c-4469-aa87-059cbf8d91e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:32.734328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03286d-217c-4469-aa87-059cbf8d91e7 None admin] PATH_INFO: `/v3/projects/947277c4458645899287e6818947c076/users/53e79d9fb2cd452d8d999a84b6425959/roles/8ff1727b65104e5fb97333d477196d46` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:32.751237 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03286d-217c-4469-aa87-059cbf8d91e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=947277c4458645899287e6818947c076, user_id=53e79d9fb2cd452d8d999a84b6425959, role_id=8ff1727b65104e5fb97333d477196d46)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:32.753073 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03286d-217c-4469-aa87-059cbf8d91e7 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:32.796234 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa6031-dd0d-46a6-b5ee-856ab8f19b60 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:32.831324 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f8fb2-bc8c-42ec-bc1a-6bcba83e8c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:32.831993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f8fb2-bc8c-42ec-bc1a-6bcba83e8c60 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:32.832273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f8fb2-bc8c-42ec-bc1a-6bcba83e8c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:32.832540 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f8fb2-bc8c-42ec-bc1a-6bcba83e8c60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:32.843469 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f8fb2-bc8c-42ec-bc1a-6bcba83e8c60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:32.844934 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f8fb2-bc8c-42ec-bc1a-6bcba83e8c60 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:32.855054 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 312/613] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Sep 10 23:42:32 2022] PUT /identity/v3/projects/947277c4458645899287e6818947c076/users/53e79d9fb2cd452d8d999a84b6425959/roles/8ff1727b65104e5fb97333d477196d46 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:42:32.857027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03286d-217c-4469-aa87-059cbf8d91e7 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:32.863665 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-513f8fb2-bc8c-42ec-bc1a-6bcba83e8c60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:32.863665 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:32.863665 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:32.863665 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:32.863665 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:32.863665 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:32.863665 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:32.863665 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:32.863665 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:32.863665 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:32.863665 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:32.863665 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:32.863665 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:32.863665 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:32.863665 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:32.863665 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:32.864708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:32.864708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:32.864708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:32.864708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:32.864708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:32.864708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:32.864708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:32.864708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:32.864708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:32.864708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:32.864708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:32.864708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:32.864708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:32.864708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:32.864708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:32.864708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:32.864708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:32.865602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:32.865602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:32.865602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:32.872276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 302/614] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:42:32.878258 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f8fb2-bc8c-42ec-bc1a-6bcba83e8c60 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:32.885640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:32.982828 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:32.986647 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03286d-217c-4469-aa87-059cbf8d91e7 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:33.024579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e45fffa-1290-4ff5-a4c1-a26c71714668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:33.025642 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45fffa-1290-4ff5-a4c1-a26c71714668 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:33.026023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45fffa-1290-4ff5-a4c1-a26c71714668 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:33.026398 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45fffa-1290-4ff5-a4c1-a26c71714668 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:33.039471 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e45fffa-1290-4ff5-a4c1-a26c71714668 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:33.045192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-5e45fffa-1290-4ff5-a4c1-a26c71714668 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:33.045192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:33.045192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:33.045192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:33.045192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:33.045192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:33.045192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:33.045192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:33.045192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:33.045192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:33.045192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:33.045192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:33.045192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:33.045192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:33.045192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:33.045192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:33.046342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:33.046342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:33.046342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:33.046342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:33.046342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:33.046342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:33.046342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:33.046342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:33.046342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:33.046342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:33.046342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:33.046342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:33.046342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:33.046342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:33.046342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:33.046342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:33.047241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:33.047241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:33.047241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:33.047241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:33.047241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:33.047241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:33.047241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:33.047241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:33.047241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:33.047241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:33.047241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:33.047241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:33.047878 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 313/615] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Sep 10 23:42:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:33.054140 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e45fffa-1290-4ff5-a4c1-a26c71714668 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:33.137011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f8fb2-bc8c-42ec-bc1a-6bcba83e8c60 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:33.180334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d095af-d89a-4754-aaa1-dce2194ebc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:33.181177 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d095af-d89a-4754-aaa1-dce2194ebc1d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:33.181603 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d095af-d89a-4754-aaa1-dce2194ebc1d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:33.182295 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d095af-d89a-4754-aaa1-dce2194ebc1d None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:33.183114 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d095af-d89a-4754-aaa1-dce2194ebc1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:33.183579 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d095af-d89a-4754-aaa1-dce2194ebc1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:33.185375 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d095af-d89a-4754-aaa1-dce2194ebc1d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:33.200261 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 303/616] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:33.206047 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d095af-d89a-4754-aaa1-dce2194ebc1d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:33.228341 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e45fffa-1290-4ff5-a4c1-a26c71714668 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:33.313535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f590501-a54c-4e92-8e0b-c1072ee1f68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:33.314149 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f590501-a54c-4e92-8e0b-c1072ee1f68a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:33.314423 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f590501-a54c-4e92-8e0b-c1072ee1f68a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:33.318723 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f590501-a54c-4e92-8e0b-c1072ee1f68a None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:33.318723 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f590501-a54c-4e92-8e0b-c1072ee1f68a None admin] RBAC: Adding query filter params (name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:33.318723 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f590501-a54c-4e92-8e0b-c1072ee1f68a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:33.318723 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f590501-a54c-4e92-8e0b-c1072ee1f68a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:33.329939 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 314/617] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Sep 10 23:42:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:42:33.330290 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:33.335792 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f590501-a54c-4e92-8e0b-c1072ee1f68a None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:33.381355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d095af-d89a-4754-aaa1-dce2194ebc1d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:33.384742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:33.425806 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690b08e-8edb-455a-ba14-d8c4e4c4ff1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:33.426702 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690b08e-8edb-455a-ba14-d8c4e4c4ff1b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:33.427063 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690b08e-8edb-455a-ba14-d8c4e4c4ff1b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:33.427440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690b08e-8edb-455a-ba14-d8c4e4c4ff1b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:33.436256 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7690b08e-8edb-455a-ba14-d8c4e4c4ff1b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:33.437896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7690b08e-8edb-455a-ba14-d8c4e4c4ff1b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:33.454069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-7690b08e-8edb-455a-ba14-d8c4e4c4ff1b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 10 23:42:33.454069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:33.454069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:33.454069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:33.454069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:33.454069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:33.454069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:33.454069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:33.454069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:33.454069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:33.454069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:33.454069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:33.454069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:42:33.454069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:42:33.454069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:42:33.454069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:42:33.455441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:33.455441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:33.455441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:42:33.455441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:33.455441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:42:33.455441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:33.455441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:33.455441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:33.455441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:33.455441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:33.455441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:33.455441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:33.455441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:33.455441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:33.455441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:33.455441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:33.455441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:33.455441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:33.456860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:33.456860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:33.456860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:42:33.456860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:42:33.456860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:42:33.456860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:42:33.456860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:42:33.456860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:42:33.456860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 10 23:42:33.456860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:33.458367 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 304/618] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:42:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:33.464008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690b08e-8edb-455a-ba14-d8c4e4c4ff1b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:33.530029 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f590501-a54c-4e92-8e0b-c1072ee1f68a None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:33.601769 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad343c-9ece-42c6-9049-e332aa8f06d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:33.602402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad343c-9ece-42c6-9049-e332aa8f06d8 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:33.602692 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad343c-9ece-42c6-9049-e332aa8f06d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:33.602964 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad343c-9ece-42c6-9049-e332aa8f06d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:33.625928 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad343c-9ece-42c6-9049-e332aa8f06d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:33.627431 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad343c-9ece-42c6-9049-e332aa8f06d8 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:33.657023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690b08e-8edb-455a-ba14-d8c4e4c4ff1b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:33.663589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-29ad343c-9ece-42c6-9049-e332aa8f06d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:33.663589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:33.663589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:33.663589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:33.663589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:33.663589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:33.663589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:33.663589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:33.663589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:33.663589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:33.663589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:33.663589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:33.663589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:33.663589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:33.663589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:33.663589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:33.664546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:33.664546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:33.664546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:33.664546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:33.664546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:33.664546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:33.664546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:33.664546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:33.664546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:33.664546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:33.664546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:33.664546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:33.664546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:33.664546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:33.664546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:33.664546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:33.664546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:33.665416 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:33.665416 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:33.665416 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:33.666030 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 315/619] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:33.676778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad343c-9ece-42c6-9049-e332aa8f06d8 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:33.709823 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:33.736140 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd0eaa2-c105-44c6-ac09-dcaad2b7fa87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:33.736741 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0eaa2-c105-44c6-ac09-dcaad2b7fa87 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:33.737019 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0eaa2-c105-44c6-ac09-dcaad2b7fa87 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:33.737288 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0eaa2-c105-44c6-ac09-dcaad2b7fa87 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:33.738020 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd0eaa2-c105-44c6-ac09-dcaad2b7fa87 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:33.738320 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd0eaa2-c105-44c6-ac09-dcaad2b7fa87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:33.739770 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd0eaa2-c105-44c6-ac09-dcaad2b7fa87 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:33.784143 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 305/620] 10.222.0.17 () {56 vars in 1135 bytes} [Sat Sep 10 23:42:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:33.788254 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd0eaa2-c105-44c6-ac09-dcaad2b7fa87 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:33.818698 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:33.880320 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad343c-9ece-42c6-9049-e332aa8f06d8 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:33.962067 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213f9d9-01fe-4191-a4e2-22fbcee174b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:33.962734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213f9d9-01fe-4191-a4e2-22fbcee174b7 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:33.963012 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213f9d9-01fe-4191-a4e2-22fbcee174b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:33.963280 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213f9d9-01fe-4191-a4e2-22fbcee174b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:33.963921 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7213f9d9-01fe-4191-a4e2-22fbcee174b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:33.964220 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7213f9d9-01fe-4191-a4e2-22fbcee174b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:33.966518 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7213f9d9-01fe-4191-a4e2-22fbcee174b7 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:33.966914 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd0eaa2-c105-44c6-ac09-dcaad2b7fa87 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:33.994051 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 316/621] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:34.003710 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213f9d9-01fe-4191-a4e2-22fbcee174b7 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:34.025608 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37879866-9c70-4fbe-924a-e893743147c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:34.026251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37879866-9c70-4fbe-924a-e893743147c4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:34.026528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37879866-9c70-4fbe-924a-e893743147c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:34.030092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37879866-9c70-4fbe-924a-e893743147c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:34.056457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37879866-9c70-4fbe-924a-e893743147c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:34.063647 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37879866-9c70-4fbe-924a-e893743147c4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:34.092578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-37879866-9c70-4fbe-924a-e893743147c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:34.092578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:34.092578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:34.092578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:34.092578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:34.092578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:34.092578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:34.092578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:34.092578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:34.092578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:34.092578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:34.092578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:34.092578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:34.092578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:34.092578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:34.092578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:34.093551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:34.093551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:34.093551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:34.093551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:34.093551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:34.093551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:34.093551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:34.093551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:34.093551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:34.093551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:34.093551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:34.093551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:34.093551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:34.093551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:34.093551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:34.093551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:34.093551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:34.094420 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:34.094420 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:34.094420 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:34.094420 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 306/622] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:34.107954 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37879866-9c70-4fbe-924a-e893743147c4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:34.150785 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213f9d9-01fe-4191-a4e2-22fbcee174b7 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:34.166528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:34.186695 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:34.195590 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081e87f-a98b-49fd-8063-d49b64a2f90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:34.196933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081e87f-a98b-49fd-8063-d49b64a2f90d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:34.197221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081e87f-a98b-49fd-8063-d49b64a2f90d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:34.197492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081e87f-a98b-49fd-8063-d49b64a2f90d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:34.207392 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081e87f-a98b-49fd-8063-d49b64a2f90d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:34.208823 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081e87f-a98b-49fd-8063-d49b64a2f90d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:34.220686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-d081e87f-a98b-49fd-8063-d49b64a2f90d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 23:42:34.220686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:34.220686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:34.220686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:34.220686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:34.220686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:34.220686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:34.220686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:34.220686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:34.220686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:34.220686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:34.220686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:34.220686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:42:34.220686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:42:34.220686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:42:34.220686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:42:34.221856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:34.221856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:34.221856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:42:34.221856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:34.221856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:42:34.221856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:34.221856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:34.221856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:34.221856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:34.221856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:34.221856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:34.221856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:34.221856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:34.221856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:34.221856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:34.221856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:34.221856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:34.221856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:34.223035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:34.223035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:34.223035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:42:34.223035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:42:34.223035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:42:34.223035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:42:34.223035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:42:34.223035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:42:34.223035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 23:42:34.223035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:34.223035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 317/623] 10.222.0.17 () {56 vars in 1093 bytes} [Sat Sep 10 23:42:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:34.227883 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2450bd-9909-4118-95a0-e95ec80d6bac None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:34.227883 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2450bd-9909-4118-95a0-e95ec80d6bac None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:34.228120 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2450bd-9909-4118-95a0-e95ec80d6bac None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:34.230820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 318/624] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:34.232026 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2450bd-9909-4118-95a0-e95ec80d6bac None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:34.264001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37879866-9c70-4fbe-924a-e893743147c4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:34.312144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1597b-e2c4-49d7-a614-749ae88fb366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:34.312755 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1597b-e2c4-49d7-a614-749ae88fb366 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:34.313035 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1597b-e2c4-49d7-a614-749ae88fb366 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:34.313309 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1597b-e2c4-49d7-a614-749ae88fb366 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:34.314118 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1597b-e2c4-49d7-a614-749ae88fb366 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:34.315276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1597b-e2c4-49d7-a614-749ae88fb366 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:34.316683 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1597b-e2c4-49d7-a614-749ae88fb366 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:34.332122 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 307/625] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:42:34.333774 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bd30d-5c6c-435b-98f4-830b4388c7e3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:34.334062 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bd30d-5c6c-435b-98f4-830b4388c7e3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:34.334336 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bd30d-5c6c-435b-98f4-830b4388c7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:34.350276 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2450bd-9909-4118-95a0-e95ec80d6bac None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:34.357544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1b8bd30d-5c6c-435b-98f4-830b4388c7e3 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:34.358779 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1b8bd30d-5c6c-435b-98f4-830b4388c7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:34.392659 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381d235-9ce8-420c-a49f-c9aff5e7b428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:34.393407 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381d235-9ce8-420c-a49f-c9aff5e7b428 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:34.393801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381d235-9ce8-420c-a49f-c9aff5e7b428 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:34.394165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381d235-9ce8-420c-a49f-c9aff5e7b428 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:34.395022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9381d235-9ce8-420c-a49f-c9aff5e7b428 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:34.395457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9381d235-9ce8-420c-a49f-c9aff5e7b428 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:34.397031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9381d235-9ce8-420c-a49f-c9aff5e7b428 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:34.421098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 319/626] 10.222.0.17 () {56 vars in 1133 bytes} [Sat Sep 10 23:42:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:34.422132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381d235-9ce8-420c-a49f-c9aff5e7b428 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:34.441420 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 308/627] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:34.442345 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8bd30d-5c6c-435b-98f4-830b4388c7e3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:34.482542 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:34.511016 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:34.601308 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381d235-9ce8-420c-a49f-c9aff5e7b428 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:34.645543 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8bd30d-5c6c-435b-98f4-830b4388c7e3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:34.663321 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad9226-1b1d-44dc-9ad0-85e2481eb00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:34.664230 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad9226-1b1d-44dc-9ad0-85e2481eb00d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:34.664668 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad9226-1b1d-44dc-9ad0-85e2481eb00d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:34.665145 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad9226-1b1d-44dc-9ad0-85e2481eb00d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:34.676883 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad9226-1b1d-44dc-9ad0-85e2481eb00d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:34.680028 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad9226-1b1d-44dc-9ad0-85e2481eb00d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:34.707847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-7fad9226-1b1d-44dc-9ad0-85e2481eb00d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:34.707847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:34.707847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:34.707847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:34.707847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:34.707847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:34.707847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:34.707847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:34.707847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:34.707847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:34.707847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:34.707847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:34.707847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:42:34.707847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:42:34.707847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:42:34.709004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:42:34.709004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:34.709004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:34.709004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:34.709004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:34.709004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:34.709004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:34.709004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:34.709004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:34.709004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:34.709004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:34.709004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:34.709004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:34.709004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:34.709004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:34.709004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:34.709004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:34.710158 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:42:34.710158 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:42:34.710158 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:42:34.710158 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:42:34.710158 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:42:34.710158 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:42:34.710158 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:34.710158 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:34.712917 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 320/628] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:34.718538 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad9226-1b1d-44dc-9ad0-85e2481eb00d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:34.721719 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832d93b-6548-43a0-af23-cf80e80fbfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:34.722756 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832d93b-6548-43a0-af23-cf80e80fbfb9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:34.723314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832d93b-6548-43a0-af23-cf80e80fbfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:34.723950 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832d93b-6548-43a0-af23-cf80e80fbfb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:34.831701 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d832d93b-6548-43a0-af23-cf80e80fbfb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:34.833144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d832d93b-6548-43a0-af23-cf80e80fbfb9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:34.859208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:34.862013 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad9226-1b1d-44dc-9ad0-85e2481eb00d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:34.871029 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:34.899361 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db950532-2d71-4431-aa97-6dce02a132a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:34.900189 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db950532-2d71-4431-aa97-6dce02a132a9 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:34.900496 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db950532-2d71-4431-aa97-6dce02a132a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:34.900936 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db950532-2d71-4431-aa97-6dce02a132a9 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:34.901657 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db950532-2d71-4431-aa97-6dce02a132a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:34.902088 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db950532-2d71-4431-aa97-6dce02a132a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:34.904005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db950532-2d71-4431-aa97-6dce02a132a9 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:34.919148 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 321/629] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:42:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:34.919148 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db950532-2d71-4431-aa97-6dce02a132a9 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:34.950467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 309/630] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:42:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:42:34.950467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd8d94-d4dc-48c3-b1ba-1d575e99b712 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:34.950467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd8d94-d4dc-48c3-b1ba-1d575e99b712 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:34.950467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd8d94-d4dc-48c3-b1ba-1d575e99b712 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:34.950467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 310/631] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:34.950467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd8d94-d4dc-48c3-b1ba-1d575e99b712 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:35.034823 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db950532-2d71-4431-aa97-6dce02a132a9 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:35.063966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd8d94-d4dc-48c3-b1ba-1d575e99b712 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:35.070786 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864efe3-2af6-4ba6-bbb6-5a5d11c25dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:35.072013 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864efe3-2af6-4ba6-bbb6-5a5d11c25dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:35.072455 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864efe3-2af6-4ba6-bbb6-5a5d11c25dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:35.072879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864efe3-2af6-4ba6-bbb6-5a5d11c25dc2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:35.086021 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d864efe3-2af6-4ba6-bbb6-5a5d11c25dc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:35.087841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:35.094142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-d864efe3-2af6-4ba6-bbb6-5a5d11c25dc2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:35.094142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:35.094142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:35.094142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:35.094142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:35.094142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:35.094142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:35.094142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:35.094142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:35.094142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:35.094142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:35.094142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:35.094142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:35.094142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:35.094142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:35.094142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:35.101241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:35.101241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:35.101241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:35.101241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:35.101241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:35.101241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:35.101241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:35.101241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:35.101241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:35.101241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:35.101241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:35.101241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:35.101241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:35.101241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:35.101241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:35.101241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:35.102243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:35.102243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:35.102243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:35.102243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:35.102243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:35.102243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:35.102243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:35.102243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:35.102243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:35.102243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:35.102243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:35.102243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:35.102243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 322/632] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Sep 10 23:42:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:35.102243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864efe3-2af6-4ba6-bbb6-5a5d11c25dc2 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:35.103167 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73dcef-7d00-4005-bbf6-7de5ab8fbaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:35.103702 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73dcef-7d00-4005-bbf6-7de5ab8fbaaa None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:35.103975 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73dcef-7d00-4005-bbf6-7de5ab8fbaaa None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:35.104244 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73dcef-7d00-4005-bbf6-7de5ab8fbaaa None admin] PATH_INFO: `/v3/projects/947277c4458645899287e6818947c076/users/564e65585e9543e286b6041af870cf41/roles/8ff1727b65104e5fb97333d477196d46` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:35.121322 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e73dcef-7d00-4005-bbf6-7de5ab8fbaaa None admin] RBAC: Authorizing `identity:create_grant(project_id=947277c4458645899287e6818947c076, user_id=564e65585e9543e286b6041af870cf41, role_id=8ff1727b65104e5fb97333d477196d46)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:35.122792 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e73dcef-7d00-4005-bbf6-7de5ab8fbaaa None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:35.132012 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:35.207112 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864efe3-2af6-4ba6-bbb6-5a5d11c25dc2 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:35.208154 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 311/633] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Sep 10 23:42:34 2022] PUT /identity/v3/projects/947277c4458645899287e6818947c076/users/564e65585e9543e286b6041af870cf41/roles/8ff1727b65104e5fb97333d477196d46 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:42:35.221277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb815543-d8a3-448a-98a2-2fc07908682d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:35.221655 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb815543-d8a3-448a-98a2-2fc07908682d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:35.222048 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb815543-d8a3-448a-98a2-2fc07908682d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:35.247165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c61cb7-1533-4210-bffe-ea00d0a4b95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:35.247850 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c61cb7-1533-4210-bffe-ea00d0a4b95b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:35.248118 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c61cb7-1533-4210-bffe-ea00d0a4b95b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:35.254514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c61cb7-1533-4210-bffe-ea00d0a4b95b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:35.261690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-eb815543-d8a3-448a-98a2-2fc07908682d None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:35.264869 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-eb815543-d8a3-448a-98a2-2fc07908682d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:35.268286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c61cb7-1533-4210-bffe-ea00d0a4b95b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:35.270155 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c61cb7-1533-4210-bffe-ea00d0a4b95b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:35.287367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-36c61cb7-1533-4210-bffe-ea00d0a4b95b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:35.287367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:35.287367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:35.287367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:35.287367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:35.287367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:35.287367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:35.287367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:35.287367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:35.287367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:35.287367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:35.287367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:35.287367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:35.287367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:35.287367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:35.287367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:35.288578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:35.288578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:35.288578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:35.288578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:35.288578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:35.288578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:35.288578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:35.288578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:35.288578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:35.288578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:35.288578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:35.288578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:35.288578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:35.288578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:35.288578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:35.288578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:35.288578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:35.289652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:35.289652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:35.289652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:35.293077 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 323/634] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:35.294090 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c61cb7-1533-4210-bffe-ea00d0a4b95b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:35.357912 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 312/635] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:35.359533 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035f5aa-ef52-40f6-a5d3-b61806d8b829 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:35.359820 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035f5aa-ef52-40f6-a5d3-b61806d8b829 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:35.360093 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035f5aa-ef52-40f6-a5d3-b61806d8b829 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:35.361640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 313/636] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:35.362171 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035f5aa-ef52-40f6-a5d3-b61806d8b829 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:35.401046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:35.431387 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c61cb7-1533-4210-bffe-ea00d0a4b95b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:35.470750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0640d3b8-2bea-45ca-a26e-dcae81c15e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:35.471639 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640d3b8-2bea-45ca-a26e-dcae81c15e9a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:35.472061 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640d3b8-2bea-45ca-a26e-dcae81c15e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:35.472477 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640d3b8-2bea-45ca-a26e-dcae81c15e9a None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:35.473278 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0640d3b8-2bea-45ca-a26e-dcae81c15e9a None admin] RBAC: Adding query filter params (name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:35.473735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0640d3b8-2bea-45ca-a26e-dcae81c15e9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:35.475303 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0640d3b8-2bea-45ca-a26e-dcae81c15e9a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:35.484371 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 324/637] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Sep 10 23:42:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:35.485747 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bd236-6b8d-44c3-9ab9-af02ffcc70e1 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:35.486161 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bd236-6b8d-44c3-9ab9-af02ffcc70e1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:35.486613 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bd236-6b8d-44c3-9ab9-af02ffcc70e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:35.489069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035f5aa-ef52-40f6-a5d3-b61806d8b829 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:35.491073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:35.506286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-f07bd236-6b8d-44c3-9ab9-af02ffcc70e1 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:35.507734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f07bd236-6b8d-44c3-9ab9-af02ffcc70e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:35.528369 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27317fdb-e601-4077-bf18-dc361efa078f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:35.529405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27317fdb-e601-4077-bf18-dc361efa078f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:35.529859 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27317fdb-e601-4077-bf18-dc361efa078f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:35.530229 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27317fdb-e601-4077-bf18-dc361efa078f None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:35.531117 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27317fdb-e601-4077-bf18-dc361efa078f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:35.537915 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27317fdb-e601-4077-bf18-dc361efa078f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:35.537915 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27317fdb-e601-4077-bf18-dc361efa078f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:35.550958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 314/638] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:35.551980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27317fdb-e601-4077-bf18-dc361efa078f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:35.582916 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 325/639] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:35.592035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bd236-6b8d-44c3-9ab9-af02ffcc70e1 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:35.682884 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27317fdb-e601-4077-bf18-dc361efa078f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:35.717572 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:35.720967 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af4368-7453-47ab-a7d9-3dc595a02e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:35.721780 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af4368-7453-47ab-a7d9-3dc595a02e0a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:35.722161 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af4368-7453-47ab-a7d9-3dc595a02e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:35.722532 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af4368-7453-47ab-a7d9-3dc595a02e0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:35.731132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bd236-6b8d-44c3-9ab9-af02ffcc70e1 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:35.735926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af4368-7453-47ab-a7d9-3dc595a02e0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:35.737613 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af4368-7453-47ab-a7d9-3dc595a02e0a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:35.754303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-49af4368-7453-47ab-a7d9-3dc595a02e0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:35.754303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:35.754303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:35.754303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:35.754303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:35.754303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:35.754303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:35.754303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:35.754303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:35.754303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:35.754303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:35.754303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:35.754303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:35.754303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:35.754303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:35.754303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:35.755481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:35.755481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:35.755481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:35.755481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:35.755481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:35.755481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:35.755481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:35.755481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:35.755481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:35.755481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:35.755481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:35.755481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:35.755481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:35.755481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:35.755481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:35.755481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:35.755481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:35.756643 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:35.756643 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:35.756643 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:35.757678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 315/640] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:35.758459 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af4368-7453-47ab-a7d9-3dc595a02e0a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:35.769980 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf473bae-322c-4146-8414-489f142441e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:35.771074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf473bae-322c-4146-8414-489f142441e5 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:35.771348 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf473bae-322c-4146-8414-489f142441e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:35.771612 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf473bae-322c-4146-8414-489f142441e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:35.778862 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:35.783713 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf473bae-322c-4146-8414-489f142441e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:35.786013 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf473bae-322c-4146-8414-489f142441e5 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:35.800129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-bf473bae-322c-4146-8414-489f142441e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:35.800129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:35.800129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:35.800129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:35.800129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:35.800129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:35.800129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:35.800129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:35.800129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:35.800129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:35.800129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:35.800129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:35.800129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:42:35.800129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:42:35.800129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:42:35.801039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:42:35.801039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:35.801039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:35.801039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:35.801039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:35.801039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:35.801039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:35.801039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:35.801039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:35.801039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:35.801039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:35.801039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:35.801039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:35.801039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:35.801039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:35.801039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:35.801039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:35.801907 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:42:35.801907 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:42:35.801907 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:42:35.801907 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:42:35.801907 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:42:35.801907 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:42:35.801907 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:35.801907 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:35.804913 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 326/641] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:35.805924 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf473bae-322c-4146-8414-489f142441e5 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:35.877828 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af4368-7453-47ab-a7d9-3dc595a02e0a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:35.914772 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d0a4d-1651-4f6e-824c-b24cc12166e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:35.915395 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d0a4d-1651-4f6e-824c-b24cc12166e2 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:35.915676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d0a4d-1651-4f6e-824c-b24cc12166e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:35.916013 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d0a4d-1651-4f6e-824c-b24cc12166e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:35.924094 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf473bae-322c-4146-8414-489f142441e5 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:35.927578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d0a4d-1651-4f6e-824c-b24cc12166e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:35.933098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-d80d0a4d-1651-4f6e-824c-b24cc12166e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:35.933098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:35.933098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:35.933098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:35.933098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:35.933098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:35.933098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:35.933098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:35.933098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:35.933098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:35.933098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:35.933098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:35.933098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:35.933098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:35.933098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:35.933098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:35.934734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:35.934734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:35.934734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:35.934734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:35.934734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:35.934734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:35.934734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:35.934734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:35.934734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:35.934734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:35.934734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:35.934734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:35.934734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:35.934734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:35.934734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:35.934734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:35.935598 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:35.935598 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:35.935598 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:35.935598 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:35.935598 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:35.935598 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:35.935598 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:35.935598 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:35.935598 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:35.935598 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:35.935598 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:35.935598 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:35.937000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 316/642] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Sep 10 23:42:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:35.937536 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d0a4d-1651-4f6e-824c-b24cc12166e2 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:35.961746 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:35.968021 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58db31f-e20d-48cf-b948-03141fcd6e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:35.968624 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58db31f-e20d-48cf-b948-03141fcd6e4f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:35.968906 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58db31f-e20d-48cf-b948-03141fcd6e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:35.969170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58db31f-e20d-48cf-b948-03141fcd6e4f None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:35.969809 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58db31f-e20d-48cf-b948-03141fcd6e4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:35.970103 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58db31f-e20d-48cf-b948-03141fcd6e4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:35.971509 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58db31f-e20d-48cf-b948-03141fcd6e4f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:35.983754 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 327/643] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:35.984562 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58db31f-e20d-48cf-b948-03141fcd6e4f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:36.021273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:36.047578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d0a4d-1651-4f6e-824c-b24cc12166e2 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:36.088207 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c56b2b-a610-4564-9c70-f631fd981bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:36.088815 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c56b2b-a610-4564-9c70-f631fd981bfd None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:36.089086 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c56b2b-a610-4564-9c70-f631fd981bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:36.089349 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c56b2b-a610-4564-9c70-f631fd981bfd None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:36.090580 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c56b2b-a610-4564-9c70-f631fd981bfd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:36.090921 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c56b2b-a610-4564-9c70-f631fd981bfd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:36.092401 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c56b2b-a610-4564-9c70-f631fd981bfd None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:36.094566 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58db31f-e20d-48cf-b948-03141fcd6e4f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:36.110486 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 317/644] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:42:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:36.110486 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c56b2b-a610-4564-9c70-f631fd981bfd None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:36.129452 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8c7e6-c0d1-4ea9-ae2e-8490582ade0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:36.130073 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8c7e6-c0d1-4ea9-ae2e-8490582ade0f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:36.130342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8c7e6-c0d1-4ea9-ae2e-8490582ade0f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:36.130606 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8c7e6-c0d1-4ea9-ae2e-8490582ade0f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:36.140933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d8c7e6-c0d1-4ea9-ae2e-8490582ade0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:36.146512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-f3d8c7e6-c0d1-4ea9-ae2e-8490582ade0f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:36.146512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:36.146512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:36.146512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:36.146512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:36.146512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:36.146512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:36.146512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:36.146512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:36.146512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:36.146512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:36.146512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:36.146512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:36.146512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:36.146512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:36.146512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:36.147581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:36.147581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:36.147581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:36.147581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:36.147581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:36.147581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:36.147581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:36.147581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:36.147581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:36.147581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:36.147581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:36.147581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:36.147581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:36.147581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:36.147581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:36.147581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:36.148803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:36.148803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:36.148803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:36.148803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:36.148803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:36.148803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:36.148803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:36.148803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:36.148803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:36.148803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:36.148803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:36.148803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:36.148803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 328/645] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Sep 10 23:42:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:36.148803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8c7e6-c0d1-4ea9-ae2e-8490582ade0f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:36.203166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:36.221386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c56b2b-a610-4564-9c70-f631fd981bfd None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:36.256724 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fa814-a2c3-4e6c-a717-e543fa0617cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:36.257318 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fa814-a2c3-4e6c-a717-e543fa0617cb None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:36.257590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fa814-a2c3-4e6c-a717-e543fa0617cb None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:36.257852 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fa814-a2c3-4e6c-a717-e543fa0617cb None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:36.258502 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652fa814-a2c3-4e6c-a717-e543fa0617cb None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:36.258825 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652fa814-a2c3-4e6c-a717-e543fa0617cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:36.260175 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652fa814-a2c3-4e6c-a717-e543fa0617cb None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:36.260513 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8c7e6-c0d1-4ea9-ae2e-8490582ade0f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:36.261707 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:36.273178 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 318/646] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Sep 10 23:42:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:36.273178 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fa814-a2c3-4e6c-a717-e543fa0617cb None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:36.311963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff8696-7bbb-4877-b58e-b8ddf38ef80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:36.312589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff8696-7bbb-4877-b58e-b8ddf38ef80a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:36.312863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff8696-7bbb-4877-b58e-b8ddf38ef80a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:36.313154 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff8696-7bbb-4877-b58e-b8ddf38ef80a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:36.321832 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff8696-7bbb-4877-b58e-b8ddf38ef80a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:36.323690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff8696-7bbb-4877-b58e-b8ddf38ef80a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:36.335503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-47ff8696-7bbb-4877-b58e-b8ddf38ef80a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 23:42:36.335503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:36.335503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:36.335503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:36.335503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:36.335503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:36.335503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:36.335503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:36.335503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:36.335503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:36.335503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:36.335503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:36.335503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:42:36.335503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:42:36.335503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:42:36.335503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:42:36.336620 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:36.336620 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:36.336620 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:42:36.336620 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:36.336620 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:42:36.336620 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:36.336620 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:36.336620 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:36.336620 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:36.336620 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:36.336620 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:36.336620 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:36.336620 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:36.336620 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:36.336620 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:36.336620 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:36.336620 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:36.336620 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:36.339554 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:36.339554 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:36.339554 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:42:36.339554 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:42:36.339554 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:42:36.339554 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:42:36.339554 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:42:36.339554 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:42:36.339554 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 23:42:36.339554 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:36.339554 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 329/647] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:42:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:36.339554 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff8696-7bbb-4877-b58e-b8ddf38ef80a None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:36.383099 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fa814-a2c3-4e6c-a717-e543fa0617cb None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:36.419195 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e0a06f-b541-4873-9611-206fdd3c80df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:36.419785 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0a06f-b541-4873-9611-206fdd3c80df None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:36.420055 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0a06f-b541-4873-9611-206fdd3c80df None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:36.420321 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0a06f-b541-4873-9611-206fdd3c80df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:36.421404 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e0a06f-b541-4873-9611-206fdd3c80df None admin] RBAC: Adding query filter params (role.id=8ff1727b65104e5fb97333d477196d46, scope.project.id=947277c4458645899287e6818947c076, user.id=636bfafb693044d283530063047fc0c2) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:36.421707 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e0a06f-b541-4873-9611-206fdd3c80df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:36.423483 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e0a06f-b541-4873-9611-206fdd3c80df None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:36.429266 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 319/648] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:42:36 2022] GET /identity/v3/role_assignments?user.id=636bfafb693044d283530063047fc0c2&scope.project.id=947277c4458645899287e6818947c076&role.id=8ff1727b65104e5fb97333d477196d46 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:36.430039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e0a06f-b541-4873-9611-206fdd3c80df None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:36.457648 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff8696-7bbb-4877-b58e-b8ddf38ef80a None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:36.471723 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:36.501007 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a18d4-e246-442d-a27a-9120615a237a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:36.501810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a18d4-e246-442d-a27a-9120615a237a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:36.502189 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a18d4-e246-442d-a27a-9120615a237a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:36.502509 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a18d4-e246-442d-a27a-9120615a237a None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:36.503376 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a18d4-e246-442d-a27a-9120615a237a None admin] RBAC: Adding query filter params (name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:36.503796 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a18d4-e246-442d-a27a-9120615a237a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:36.506944 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a18d4-e246-442d-a27a-9120615a237a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:36.509291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:36.519780 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 330/649] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Sep 10 23:42:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:36.521093 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a18d4-e246-442d-a27a-9120615a237a None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:36.569029 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e0a06f-b541-4873-9611-206fdd3c80df None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:36.616458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab29caa7-a8f9-45d5-95ae-cc88055ef501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:36.617082 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29caa7-a8f9-45d5-95ae-cc88055ef501 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:36.617403 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29caa7-a8f9-45d5-95ae-cc88055ef501 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:36.617783 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29caa7-a8f9-45d5-95ae-cc88055ef501 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:36.634511 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab29caa7-a8f9-45d5-95ae-cc88055ef501 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:36.635901 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab29caa7-a8f9-45d5-95ae-cc88055ef501 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:36.652966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-ab29caa7-a8f9-45d5-95ae-cc88055ef501 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:36.652966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:36.652966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:36.652966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:36.652966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:36.652966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:36.652966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:36.652966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:36.652966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:36.652966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:36.652966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:36.652966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:36.652966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:36.652966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:36.652966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:36.652966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:36.654047 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:36.654047 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:36.654047 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:36.654047 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:36.654047 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:36.654047 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:36.654047 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:36.654047 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:36.654047 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:36.654047 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:36.654047 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:36.654047 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:36.654047 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:36.654047 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:36.654047 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:36.654047 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:36.654047 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:36.655488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:36.655488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:36.655488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:36.655488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 320/650] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:36.656835 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a18d4-e246-442d-a27a-9120615a237a None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:36.657210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab29caa7-a8f9-45d5-95ae-cc88055ef501 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:36.703147 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c73aab4-7922-4b2a-86f7-9c50dd4eb4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:36.703757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73aab4-7922-4b2a-86f7-9c50dd4eb4e4 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:36.704039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73aab4-7922-4b2a-86f7-9c50dd4eb4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:36.704309 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73aab4-7922-4b2a-86f7-9c50dd4eb4e4 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:36.705032 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c73aab4-7922-4b2a-86f7-9c50dd4eb4e4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:36.705336 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c73aab4-7922-4b2a-86f7-9c50dd4eb4e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:36.706830 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c73aab4-7922-4b2a-86f7-9c50dd4eb4e4 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:36.729955 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 331/651] 10.222.0.17 () {56 vars in 1137 bytes} [Sat Sep 10 23:42:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:36.731872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c73aab4-7922-4b2a-86f7-9c50dd4eb4e4 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:36.772169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:36.789159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab29caa7-a8f9-45d5-95ae-cc88055ef501 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:36.806580 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:36.836447 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038eba3e-19a1-4faa-bf99-dd1348e5d563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:36.837471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038eba3e-19a1-4faa-bf99-dd1348e5d563 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:36.837761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038eba3e-19a1-4faa-bf99-dd1348e5d563 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:36.838033 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038eba3e-19a1-4faa-bf99-dd1348e5d563 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:36.853894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038eba3e-19a1-4faa-bf99-dd1348e5d563 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:36.854249 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c73aab4-7922-4b2a-86f7-9c50dd4eb4e4 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:36.856131 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038eba3e-19a1-4faa-bf99-dd1348e5d563 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:36.876147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-038eba3e-19a1-4faa-bf99-dd1348e5d563 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:36.876147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:36.876147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:36.876147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:36.876147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:36.876147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:36.876147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:36.876147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:36.876147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:36.876147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:36.876147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:36.876147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:36.876147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:36.876147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:36.876147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:36.876147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:36.878251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:36.878251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:36.878251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:36.878251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:36.878251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:36.878251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:36.878251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:36.878251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:36.878251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:36.878251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:36.878251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:36.878251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:36.878251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:36.878251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:36.878251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:36.878251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:36.878251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:36.879874 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:36.879874 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:36.879874 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:36.879874 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 321/652] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:36.883065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038eba3e-19a1-4faa-bf99-dd1348e5d563 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:36.904857 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45591d63-b00e-463d-9789-e635729b6312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:36.905631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45591d63-b00e-463d-9789-e635729b6312 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:36.905911 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45591d63-b00e-463d-9789-e635729b6312 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:36.906490 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45591d63-b00e-463d-9789-e635729b6312 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:36.907239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45591d63-b00e-463d-9789-e635729b6312 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:36.907543 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45591d63-b00e-463d-9789-e635729b6312 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:36.909438 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45591d63-b00e-463d-9789-e635729b6312 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:36.930610 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 332/653] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:36.931544 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45591d63-b00e-463d-9789-e635729b6312 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:36.992837 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038eba3e-19a1-4faa-bf99-dd1348e5d563 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:37.034821 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c5a92-7cfc-4717-9763-3c5bb89e0a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:37.035912 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c5a92-7cfc-4717-9763-3c5bb89e0a1b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:37.036191 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c5a92-7cfc-4717-9763-3c5bb89e0a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:37.036468 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c5a92-7cfc-4717-9763-3c5bb89e0a1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:37.049490 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:37.062692 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45591d63-b00e-463d-9789-e635729b6312 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:37.080369 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:37.098010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307f254-57a8-47ed-92ab-f5cadfe2e93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:37.099334 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307f254-57a8-47ed-92ab-f5cadfe2e93e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:37.100207 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307f254-57a8-47ed-92ab-f5cadfe2e93e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:37.100806 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307f254-57a8-47ed-92ab-f5cadfe2e93e None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:37.101680 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3307f254-57a8-47ed-92ab-f5cadfe2e93e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:37.102308 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3307f254-57a8-47ed-92ab-f5cadfe2e93e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:37.104119 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3307f254-57a8-47ed-92ab-f5cadfe2e93e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:37.117582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 333/654] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:37.118858 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307f254-57a8-47ed-92ab-f5cadfe2e93e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:37.134771 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c5a92-7cfc-4717-9763-3c5bb89e0a1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:37.137360 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c5a92-7cfc-4717-9763-3c5bb89e0a1b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:37.229204 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307f254-57a8-47ed-92ab-f5cadfe2e93e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:37.266529 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c6932-b7ef-49b6-a926-37607399578d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:37.267548 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c6932-b7ef-49b6-a926-37607399578d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:37.268038 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c6932-b7ef-49b6-a926-37607399578d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:37.268519 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c6932-b7ef-49b6-a926-37607399578d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:37.276425 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c6932-b7ef-49b6-a926-37607399578d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:37.277455 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 322/655] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:42:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:42:37.278384 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c5a92-7cfc-4717-9763-3c5bb89e0a1b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:37.279226 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c6932-b7ef-49b6-a926-37607399578d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:37.304312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-555c6932-b7ef-49b6-a926-37607399578d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 10 23:42:37.304312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:37.304312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:37.304312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:37.304312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:37.304312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:37.304312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:37.304312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:37.304312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:37.304312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:37.304312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:37.304312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:37.304312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:42:37.304312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:42:37.304312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:42:37.304312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:42:37.305277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:37.305277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:37.305277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:42:37.305277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:37.305277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:42:37.305277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:37.305277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:37.305277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:37.305277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:37.305277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:37.305277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:37.305277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:37.305277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:37.305277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:37.305277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:37.305277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:37.305277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:37.305277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:37.306246 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:37.306246 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:37.306246 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:42:37.306246 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:42:37.306246 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:42:37.306246 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:42:37.306246 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:42:37.306246 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:42:37.306246 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 10 23:42:37.306246 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:37.308936 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 334/656] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:42:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:37.310035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c6932-b7ef-49b6-a926-37607399578d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:37.327739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:37.360925 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:37.403786 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c5a92-7cfc-4717-9763-3c5bb89e0a1b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:37.442181 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbd7b6-0b49-483b-95a1-01e6c99d2d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:37.443210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbd7b6-0b49-483b-95a1-01e6c99d2d1f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:37.443705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbd7b6-0b49-483b-95a1-01e6c99d2d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:37.444181 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbd7b6-0b49-483b-95a1-01e6c99d2d1f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:37.453712 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbbd7b6-0b49-483b-95a1-01e6c99d2d1f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:37.454117 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c6932-b7ef-49b6-a926-37607399578d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:37.455901 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbbd7b6-0b49-483b-95a1-01e6c99d2d1f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:37.468634 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-4cbbd7b6-0b49-483b-95a1-01e6c99d2d1f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 10 23:42:37.468634 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:37.468634 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:37.468634 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:37.468634 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:37.468634 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:37.468634 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:37.468634 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:37.468634 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:37.468634 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:37.468634 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:37.468634 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:37.468634 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:42:37.468634 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:42:37.468634 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:42:37.468634 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:42:37.469675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:37.469675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:37.469675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:42:37.469675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:37.469675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:42:37.469675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:37.469675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:37.469675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:37.469675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:37.469675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:37.469675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:37.469675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:37.469675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:37.469675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:37.469675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:37.469675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:37.469675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:37.469675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:37.470592 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:37.470592 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:37.470592 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:42:37.470592 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:42:37.470592 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:42:37.470592 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:42:37.470592 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:42:37.470592 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:42:37.470592 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 10 23:42:37.470592 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:37.473293 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 323/657] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Sep 10 23:42:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:37.474287 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbd7b6-0b49-483b-95a1-01e6c99d2d1f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:37.498918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e64425-289d-47a2-bce5-f97f608e26ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:37.499921 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e64425-289d-47a2-bce5-f97f608e26ef None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:37.500446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e64425-289d-47a2-bce5-f97f608e26ef None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:37.500973 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e64425-289d-47a2-bce5-f97f608e26ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:37.513004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e64425-289d-47a2-bce5-f97f608e26ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:37.514473 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e64425-289d-47a2-bce5-f97f608e26ef None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:37.529663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-20e64425-289d-47a2-bce5-f97f608e26ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:37.529663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:37.529663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:37.529663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:37.529663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:37.529663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:37.529663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:37.529663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:37.529663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:37.529663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:37.529663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:37.529663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:37.529663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:37.529663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:37.529663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:37.529663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:37.530743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:37.530743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:37.530743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:37.530743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:37.530743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:37.530743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:37.530743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:37.530743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:37.530743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:37.530743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:37.530743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:37.530743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:37.530743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:37.530743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:37.530743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:37.530743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:37.530743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:37.531617 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:37.531617 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:37.531617 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:37.531617 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 335/658] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:37.531617 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e64425-289d-47a2-bce5-f97f608e26ef None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:37.594258 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:37.597714 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbd7b6-0b49-483b-95a1-01e6c99d2d1f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:37.621861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:37.632791 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6db16-f715-4112-9c3d-0056303f5e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:37.633490 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6db16-f715-4112-9c3d-0056303f5e73 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:37.638238 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6db16-f715-4112-9c3d-0056303f5e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:37.638238 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6db16-f715-4112-9c3d-0056303f5e73 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:37.638238 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6db16-f715-4112-9c3d-0056303f5e73 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:37.638238 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6db16-f715-4112-9c3d-0056303f5e73 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:37.638599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e64425-289d-47a2-bce5-f97f608e26ef None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:37.638721 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6db16-f715-4112-9c3d-0056303f5e73 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:37.661092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 324/659] 10.222.0.17 () {56 vars in 1137 bytes} [Sat Sep 10 23:42:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:37.661092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201524db-e329-47f5-b686-055900990e6c None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:37.661092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201524db-e329-47f5-b686-055900990e6c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:37.661092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201524db-e329-47f5-b686-055900990e6c None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:37.666002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 325/660] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:37 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:37.667662 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201524db-e329-47f5-b686-055900990e6c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:37.682996 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1db57-b9fe-4970-9710-41a8443c0d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:37.684086 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1db57-b9fe-4970-9710-41a8443c0d5c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:37.684492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1db57-b9fe-4970-9710-41a8443c0d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:37.685194 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1db57-b9fe-4970-9710-41a8443c0d5c None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:37.685996 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1db57-b9fe-4970-9710-41a8443c0d5c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:37.686370 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1db57-b9fe-4970-9710-41a8443c0d5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:37.687902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1db57-b9fe-4970-9710-41a8443c0d5c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:37.706762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 336/661] 10.222.0.17 () {56 vars in 1139 bytes} [Sat Sep 10 23:42:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:37.707669 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1db57-b9fe-4970-9710-41a8443c0d5c None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:37.773242 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201524db-e329-47f5-b686-055900990e6c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:37.815203 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1db57-b9fe-4970-9710-41a8443c0d5c None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:37.819078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298d711-74ae-4836-b346-722bbc39a99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:37.819683 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298d711-74ae-4836-b346-722bbc39a99f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:37.819965 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298d711-74ae-4836-b346-722bbc39a99f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:37.820247 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298d711-74ae-4836-b346-722bbc39a99f None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:37.820874 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298d711-74ae-4836-b346-722bbc39a99f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:37.821174 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298d711-74ae-4836-b346-722bbc39a99f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:37.823072 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298d711-74ae-4836-b346-722bbc39a99f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:37.838719 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 326/662] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:37.838912 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:37.838997 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa022dd8-e951-4c24-9202-9f714cbe5b19 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:37.838997 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa022dd8-e951-4c24-9202-9f714cbe5b19 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:37.838997 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa022dd8-e951-4c24-9202-9f714cbe5b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:37.856150 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fec688-0b62-44c8-ad89-5093f406c0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:37.856753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fec688-0b62-44c8-ad89-5093f406c0b9 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:37.857024 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fec688-0b62-44c8-ad89-5093f406c0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:37.857293 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fec688-0b62-44c8-ad89-5093f406c0b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:37.858582 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-fa022dd8-e951-4c24-9202-9f714cbe5b19 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:37.859808 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fa022dd8-e951-4c24-9202-9f714cbe5b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:37.898589 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:37.926411 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fec688-0b62-44c8-ad89-5093f406c0b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:37.927963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fec688-0b62-44c8-ad89-5093f406c0b9 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:37.938729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 327/663] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:37.938729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa022dd8-e951-4c24-9202-9f714cbe5b19 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:38.049009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa022dd8-e951-4c24-9202-9f714cbe5b19 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:38.073397 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 337/664] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:42:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:42:38.074993 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fec688-0b62-44c8-ad89-5093f406c0b9 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:38.095691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fda6ca-edae-4d1d-a8db-41026b6b98ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:38.096293 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fda6ca-edae-4d1d-a8db-41026b6b98ea None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:38.096569 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fda6ca-edae-4d1d-a8db-41026b6b98ea None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:38.096839 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fda6ca-edae-4d1d-a8db-41026b6b98ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:38.102111 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:38.149384 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:38.181471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fda6ca-edae-4d1d-a8db-41026b6b98ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:38.184489 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fda6ca-edae-4d1d-a8db-41026b6b98ea None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:38.204640 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fec688-0b62-44c8-ad89-5093f406c0b9 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:38.246337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d939738-6835-45f7-80a6-7da07da975fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:38.247116 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d939738-6835-45f7-80a6-7da07da975fe None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:38.247401 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d939738-6835-45f7-80a6-7da07da975fe None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:38.247673 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d939738-6835-45f7-80a6-7da07da975fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:38.255481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d939738-6835-45f7-80a6-7da07da975fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:38.257389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d939738-6835-45f7-80a6-7da07da975fe None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:38.269564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-0d939738-6835-45f7-80a6-7da07da975fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:38.269564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:38.269564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:38.269564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:38.269564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:38.269564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:38.269564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:38.269564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:38.269564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:38.269564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:38.269564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:38.269564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:38.269564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:42:38.269564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:42:38.269564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:42:38.270637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:42:38.270637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:38.270637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:38.270637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:38.270637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:38.270637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:38.270637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:38.270637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:38.270637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:38.270637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:38.270637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:38.270637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:38.270637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:38.270637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:38.270637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:38.270637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:38.270637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:38.271552 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:42:38.271552 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:42:38.271552 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:42:38.271552 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:42:38.271552 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:42:38.271552 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:42:38.271552 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:38.271552 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:38.271552 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 338/665] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:38.271552 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d939738-6835-45f7-80a6-7da07da975fe None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:38.306516 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 328/666] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:42:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:42:38.307707 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fda6ca-edae-4d1d-a8db-41026b6b98ea None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:38.361364 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:38.382722 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:38.385168 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d939738-6835-45f7-80a6-7da07da975fe None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:38.415915 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fda6ca-edae-4d1d-a8db-41026b6b98ea None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:38.426627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9ee19-22f4-47de-88df-9ae06e95d2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:38.427281 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9ee19-22f4-47de-88df-9ae06e95d2fe None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:38.427559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9ee19-22f4-47de-88df-9ae06e95d2fe None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:38.427830 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9ee19-22f4-47de-88df-9ae06e95d2fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:38.438493 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9ee19-22f4-47de-88df-9ae06e95d2fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:38.439960 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9ee19-22f4-47de-88df-9ae06e95d2fe None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:38.452377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f832719-c7a1-4bcd-80ae-97cf89bc97e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:38.453200 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f832719-c7a1-4bcd-80ae-97cf89bc97e4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:38.453666 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f832719-c7a1-4bcd-80ae-97cf89bc97e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:38.454104 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f832719-c7a1-4bcd-80ae-97cf89bc97e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:38.455467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-e0e9ee19-22f4-47de-88df-9ae06e95d2fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:38.455467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:38.455467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:38.455467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:38.455467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:38.455467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:38.455467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:38.455467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:38.455467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:38.455467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:38.455467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:38.455467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:38.455467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:38.455467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:38.455467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:38.455467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:38.456382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:38.456382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:38.456382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:38.456382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:38.456382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:38.456382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:38.456382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:38.456382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:38.456382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:38.456382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:38.456382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:38.456382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:38.456382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:38.456382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:38.456382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:38.456382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:38.456382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:38.457257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:38.457257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:38.457257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:38.457257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 339/667] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:38.457494 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9ee19-22f4-47de-88df-9ae06e95d2fe None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:38.472338 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f832719-c7a1-4bcd-80ae-97cf89bc97e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:38.478260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-6f832719-c7a1-4bcd-80ae-97cf89bc97e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:38.478260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:38.478260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:38.478260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:38.478260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:38.478260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:38.478260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:38.478260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:38.478260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:38.478260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:38.478260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:38.478260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:38.478260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:38.478260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:38.478260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:38.478260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:38.479222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:38.479222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:38.479222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:38.479222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:38.479222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:38.479222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:38.479222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:38.479222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:38.479222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:38.479222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:38.479222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:38.479222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:38.479222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:38.479222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:38.479222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:38.479222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:38.480044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:38.480044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:38.480044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:38.480044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:38.480044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:38.480044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:38.480044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:38.480044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:38.480044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:38.480044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:38.480044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:38.480044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:38.483222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 329/668] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Sep 10 23:42:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:38.484020 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f832719-c7a1-4bcd-80ae-97cf89bc97e4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:38.571397 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9ee19-22f4-47de-88df-9ae06e95d2fe None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:38.587273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f832719-c7a1-4bcd-80ae-97cf89bc97e4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:38.605936 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:38.611317 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92daa514-8c98-4432-a544-aeae372a0af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:38.615248 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92daa514-8c98-4432-a544-aeae372a0af5 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:38.615248 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92daa514-8c98-4432-a544-aeae372a0af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:38.615248 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92daa514-8c98-4432-a544-aeae372a0af5 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:38.615248 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92daa514-8c98-4432-a544-aeae372a0af5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:38.615248 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92daa514-8c98-4432-a544-aeae372a0af5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:38.615599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92daa514-8c98-4432-a544-aeae372a0af5 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:38.618776 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:38.624044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940dfc2c-1e02-4cbe-a8c0-8b1d9b542dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:38.624686 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940dfc2c-1e02-4cbe-a8c0-8b1d9b542dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:38.624970 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940dfc2c-1e02-4cbe-a8c0-8b1d9b542dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:38.625240 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940dfc2c-1e02-4cbe-a8c0-8b1d9b542dc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:38.635018 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 340/669] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:42:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:38.635018 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92daa514-8c98-4432-a544-aeae372a0af5 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:38.641150 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940dfc2c-1e02-4cbe-a8c0-8b1d9b542dc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:38.642857 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940dfc2c-1e02-4cbe-a8c0-8b1d9b542dc3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:38.661071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-940dfc2c-1e02-4cbe-a8c0-8b1d9b542dc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:38.661071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:38.661071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:38.661071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:38.661071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:38.661071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:38.661071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:38.661071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:38.661071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:38.661071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:38.661071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:38.661071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:38.661071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:38.661071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:38.661071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:38.661071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:38.662096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:38.662096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:38.662096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:38.662096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:38.662096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:38.662096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:38.662096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:38.662096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:38.662096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:38.662096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:38.662096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:38.662096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:38.662096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:38.662096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:38.662096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:38.662096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:38.662096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:38.663001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:38.663001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:38.663001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:38.663001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 330/670] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:38.663001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940dfc2c-1e02-4cbe-a8c0-8b1d9b542dc3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:38.749758 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92daa514-8c98-4432-a544-aeae372a0af5 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:38.781915 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940dfc2c-1e02-4cbe-a8c0-8b1d9b542dc3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:38.784759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42fa52-5e6b-41c4-9cd8-828442d86226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:38.785351 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42fa52-5e6b-41c4-9cd8-828442d86226 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:38.785620 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42fa52-5e6b-41c4-9cd8-828442d86226 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:38.785884 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42fa52-5e6b-41c4-9cd8-828442d86226 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:38.786513 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db42fa52-5e6b-41c4-9cd8-828442d86226 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:38.786841 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db42fa52-5e6b-41c4-9cd8-828442d86226 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:38.788184 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db42fa52-5e6b-41c4-9cd8-828442d86226 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:38.800122 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 341/671] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:38.800923 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42fa52-5e6b-41c4-9cd8-828442d86226 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:38.819200 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a66b75-7e50-41a7-8986-97ff254a23a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:38.819799 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a66b75-7e50-41a7-8986-97ff254a23a7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:38.820077 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a66b75-7e50-41a7-8986-97ff254a23a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:38.820345 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a66b75-7e50-41a7-8986-97ff254a23a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:38.821006 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a66b75-7e50-41a7-8986-97ff254a23a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:38.821304 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a66b75-7e50-41a7-8986-97ff254a23a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:38.822684 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a66b75-7e50-41a7-8986-97ff254a23a7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:38.830515 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 331/672] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Sep 10 23:42:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:38.831329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a66b75-7e50-41a7-8986-97ff254a23a7 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:38.852825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:38.878603 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:38.899464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42fa52-5e6b-41c4-9cd8-828442d86226 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:38.932569 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a66b75-7e50-41a7-8986-97ff254a23a7 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:38.935265 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba0b32-41e6-4f31-ba2a-0b2c6b831527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:38.935859 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba0b32-41e6-4f31-ba2a-0b2c6b831527 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:38.936132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba0b32-41e6-4f31-ba2a-0b2c6b831527 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:38.936396 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba0b32-41e6-4f31-ba2a-0b2c6b831527 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:38.937113 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba0b32-41e6-4f31-ba2a-0b2c6b831527 None admin] RBAC: Adding query filter params (role.id=8ff1727b65104e5fb97333d477196d46, scope.project.id=947277c4458645899287e6818947c076, user.id=c6fccf47e1ec4324983c5218ec3c96a4) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:38.937413 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba0b32-41e6-4f31-ba2a-0b2c6b831527 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:38.938845 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba0b32-41e6-4f31-ba2a-0b2c6b831527 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:38.944401 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 342/673] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:42:38 2022] GET /identity/v3/role_assignments?user.id=c6fccf47e1ec4324983c5218ec3c96a4&scope.project.id=947277c4458645899287e6818947c076&role.id=8ff1727b65104e5fb97333d477196d46 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:38.945533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e8da4-c340-423e-ab8f-3c79812deb51 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:38.945810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e8da4-c340-423e-ab8f-3c79812deb51 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:38.946076 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e8da4-c340-423e-ab8f-3c79812deb51 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:38.947270 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 343/674] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:38.947952 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e8da4-c340-423e-ab8f-3c79812deb51 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:38.983328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27900754-8ca0-4a1c-8d09-8a69195d738f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:38.984062 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27900754-8ca0-4a1c-8d09-8a69195d738f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:38.984336 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27900754-8ca0-4a1c-8d09-8a69195d738f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:38.984711 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27900754-8ca0-4a1c-8d09-8a69195d738f None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:38.985419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27900754-8ca0-4a1c-8d09-8a69195d738f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:38.985727 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27900754-8ca0-4a1c-8d09-8a69195d738f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:38.987147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27900754-8ca0-4a1c-8d09-8a69195d738f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:39.005919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 332/675] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:39.005919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27900754-8ca0-4a1c-8d09-8a69195d738f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:39.088693 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e8da4-c340-423e-ab8f-3c79812deb51 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:39.113444 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:39.127400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27900754-8ca0-4a1c-8d09-8a69195d738f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:39.135926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:39.136252 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00acd26-c228-4373-9846-2b0a0a56b160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:39.136977 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00acd26-c228-4373-9846-2b0a0a56b160 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:39.137254 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00acd26-c228-4373-9846-2b0a0a56b160 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:39.138132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00acd26-c228-4373-9846-2b0a0a56b160 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:39.146508 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00acd26-c228-4373-9846-2b0a0a56b160 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:39.148071 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00acd26-c228-4373-9846-2b0a0a56b160 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:39.161538 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-f00acd26-c228-4373-9846-2b0a0a56b160 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:39.161538 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:39.161538 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:39.161538 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:39.161538 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:39.161538 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:39.161538 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:39.161538 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:39.161538 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:39.161538 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:39.161538 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:39.161538 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:39.161538 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:42:39.161538 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:42:39.161538 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:42:39.162615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:42:39.162615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:39.162615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:39.162615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:39.162615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:39.162615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:39.162615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:39.162615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:39.162615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:39.162615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:39.162615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:39.162615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:39.162615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:39.162615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:39.162615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:39.162615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:39.162615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:39.163544 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:42:39.163544 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:42:39.163544 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:42:39.163544 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:42:39.163544 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:42:39.163544 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:42:39.163544 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:39.163544 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:39.163544 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 344/676] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:39.163544 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbb629-9902-4131-af21-c6dff768666c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:39.164089 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbb629-9902-4131-af21-c6dff768666c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:39.164089 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbb629-9902-4131-af21-c6dff768666c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:39.170280 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d8b6b-9d4f-4120-8fcf-f6a63afa6786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:39.171450 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d8b6b-9d4f-4120-8fcf-f6a63afa6786 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:39.172762 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d8b6b-9d4f-4120-8fcf-f6a63afa6786 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:39.173232 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d8b6b-9d4f-4120-8fcf-f6a63afa6786 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:39.183568 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-70cbb629-9902-4131-af21-c6dff768666c None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:39.184901 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-70cbb629-9902-4131-af21-c6dff768666c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:39.186152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d8b6b-9d4f-4120-8fcf-f6a63afa6786 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:39.188664 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d8b6b-9d4f-4120-8fcf-f6a63afa6786 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:39.206462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-241d8b6b-9d4f-4120-8fcf-f6a63afa6786 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:39.206462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:39.206462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:39.206462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:39.206462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:39.206462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:39.206462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:39.206462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:39.206462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:39.206462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:39.206462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:39.206462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:39.206462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:39.206462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:39.206462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:39.206462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:39.207668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:39.207668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:39.207668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:39.207668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:39.207668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:39.207668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:39.207668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:39.207668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:39.207668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:39.207668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:39.207668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:39.207668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:39.207668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:39.207668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:39.207668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:39.207668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:39.207668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:39.208737 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:39.208737 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:39.208737 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:39.208737 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 333/677] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:39.208737 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d8b6b-9d4f-4120-8fcf-f6a63afa6786 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:39.264905 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 345/678] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:39.265747 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cbb629-9902-4131-af21-c6dff768666c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:39.336560 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d8b6b-9d4f-4120-8fcf-f6a63afa6786 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:39.382190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383754b-eb77-467c-b481-ddc4977f31bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:39.383745 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383754b-eb77-467c-b481-ddc4977f31bc None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:39.384166 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383754b-eb77-467c-b481-ddc4977f31bc None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:39.384575 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383754b-eb77-467c-b481-ddc4977f31bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:39.389879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cbb629-9902-4131-af21-c6dff768666c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:39.394702 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383754b-eb77-467c-b481-ddc4977f31bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:39.396324 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383754b-eb77-467c-b481-ddc4977f31bc None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:39.398459 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:39.413063 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-a383754b-eb77-467c-b481-ddc4977f31bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:39.413063 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:39.413063 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:39.413063 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:39.413063 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:39.413063 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:39.413063 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:39.413063 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:39.413063 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:39.413063 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:39.413063 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:39.413063 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:39.413063 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:42:39.413063 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:42:39.413063 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:42:39.425088 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:42:39.425088 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:39.425088 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:39.425088 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:39.425088 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:39.425088 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:39.425088 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:39.425088 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:39.425088 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:39.425088 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:39.425088 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:39.425088 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:39.425088 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:39.425088 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:39.425088 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:39.425088 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:39.425088 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:39.426148 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:42:39.426148 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:42:39.426148 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:42:39.426148 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:42:39.426148 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:42:39.426148 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:42:39.426148 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:39.426148 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:39.426148 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 334/679] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:39.426148 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383754b-eb77-467c-b481-ddc4977f31bc None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:39.442067 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5026e114-309f-4890-b7ac-b86d1ed953b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:39.443425 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026e114-309f-4890-b7ac-b86d1ed953b5 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:39.444383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026e114-309f-4890-b7ac-b86d1ed953b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:39.444849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026e114-309f-4890-b7ac-b86d1ed953b5 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:39.445754 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5026e114-309f-4890-b7ac-b86d1ed953b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:39.446250 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5026e114-309f-4890-b7ac-b86d1ed953b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:39.447914 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5026e114-309f-4890-b7ac-b86d1ed953b5 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:39.451987 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:39.461746 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 346/680] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:42:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:39.462919 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5026e114-309f-4890-b7ac-b86d1ed953b5 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:39.537991 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383754b-eb77-467c-b481-ddc4977f31bc None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:39.570421 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5026e114-309f-4890-b7ac-b86d1ed953b5 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:39.573163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d05a9b-9e7b-4ee5-b259-78594d163f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:39.573765 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d05a9b-9e7b-4ee5-b259-78594d163f15 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:39.574220 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d05a9b-9e7b-4ee5-b259-78594d163f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:39.574502 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d05a9b-9e7b-4ee5-b259-78594d163f15 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:39.575180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d05a9b-9e7b-4ee5-b259-78594d163f15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:39.575489 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d05a9b-9e7b-4ee5-b259-78594d163f15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:39.576895 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d05a9b-9e7b-4ee5-b259-78594d163f15 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:39.588462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 335/681] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:39.589159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d05a9b-9e7b-4ee5-b259-78594d163f15 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:39.610237 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ddec4e-4038-412a-9534-1df7fa9472f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:39.611398 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ddec4e-4038-412a-9534-1df7fa9472f2 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:39.611772 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ddec4e-4038-412a-9534-1df7fa9472f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:39.612136 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ddec4e-4038-412a-9534-1df7fa9472f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:39.612815 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ddec4e-4038-412a-9534-1df7fa9472f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:39.613115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ddec4e-4038-412a-9534-1df7fa9472f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:39.614513 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ddec4e-4038-412a-9534-1df7fa9472f2 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:39.628063 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 347/682] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:42:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:39.633420 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ddec4e-4038-412a-9534-1df7fa9472f2 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:39.664922 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:39.674094 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:39.697114 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d05a9b-9e7b-4ee5-b259-78594d163f15 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:39.750334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0aa4b-3645-458f-aa13-08604dae4131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:39.750973 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0aa4b-3645-458f-aa13-08604dae4131 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:39.751240 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0aa4b-3645-458f-aa13-08604dae4131 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:39.751505 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0aa4b-3645-458f-aa13-08604dae4131 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:39.752349 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0aa4b-3645-458f-aa13-08604dae4131 None admin] RBAC: Adding query filter params (role.id=8ff1727b65104e5fb97333d477196d46, scope.project.id=947277c4458645899287e6818947c076, user.id=2cb18199dc8b42589184042556e98983) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:39.752659 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0aa4b-3645-458f-aa13-08604dae4131 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:39.753193 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ddec4e-4038-412a-9534-1df7fa9472f2 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:39.754104 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0aa4b-3645-458f-aa13-08604dae4131 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:39.760282 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 336/683] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:42:39 2022] GET /identity/v3/role_assignments?user.id=2cb18199dc8b42589184042556e98983&scope.project.id=947277c4458645899287e6818947c076&role.id=8ff1727b65104e5fb97333d477196d46 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:39.765269 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0aa4b-3645-458f-aa13-08604dae4131 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:39.796221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85933310-a6ee-4064-a182-4f1040ad3290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:39.797429 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85933310-a6ee-4064-a182-4f1040ad3290 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:39.797786 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85933310-a6ee-4064-a182-4f1040ad3290 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:39.798139 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85933310-a6ee-4064-a182-4f1040ad3290 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:39.812847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85933310-a6ee-4064-a182-4f1040ad3290 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:39.818194 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-85933310-a6ee-4064-a182-4f1040ad3290 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:42:39.818194 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:39.818194 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:39.818194 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:39.818194 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:39.818194 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:39.818194 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:39.818194 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:39.818194 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:39.818194 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:39.818194 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:39.818194 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:39.818194 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:39.818194 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:39.818194 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:39.818194 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:39.819225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:39.819225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:39.819225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:39.819225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:39.819225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:39.819225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:39.819225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:39.819225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:39.819225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:39.819225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:39.819225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:39.819225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:39.819225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:39.819225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:39.819225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:39.819225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:39.820053 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:39.820053 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:39.820053 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:39.820053 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:39.820053 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:39.820053 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:39.820053 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:39.820053 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:39.820053 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:39.820053 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:39.820053 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:42:39.820053 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:39.820053 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 348/684] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:42:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:39.820733 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85933310-a6ee-4064-a182-4f1040ad3290 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:39.921144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0aa4b-3645-458f-aa13-08604dae4131 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:39.934967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:39.951938 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85933310-a6ee-4064-a182-4f1040ad3290 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:39.967407 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56630a90-8e72-4f71-b1b4-8dea24b490c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:39.968117 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56630a90-8e72-4f71-b1b4-8dea24b490c7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:39.969286 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56630a90-8e72-4f71-b1b4-8dea24b490c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:39.969565 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56630a90-8e72-4f71-b1b4-8dea24b490c7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:39.977834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56630a90-8e72-4f71-b1b4-8dea24b490c7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:39.979595 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56630a90-8e72-4f71-b1b4-8dea24b490c7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:39.991548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-56630a90-8e72-4f71-b1b4-8dea24b490c7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 10 23:42:39.991548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:39.991548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:39.991548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:39.991548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:39.991548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:39.991548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:39.991548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:39.991548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:39.991548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:39.991548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:39.991548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:39.991548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:42:39.991548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:42:39.991548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:42:39.991548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:42:39.992678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:39.992678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:39.992678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:42:39.992678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:39.992678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:42:39.992678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:39.992678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:39.992678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:39.992678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:39.992678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:39.992678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:39.992678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:39.992678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:39.992678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:39.992678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:39.992678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:39.992678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:39.992678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:39.993619 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:39.993619 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:39.993619 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:42:39.993619 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:42:39.993619 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:42:39.993619 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:42:39.993619 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:42:39.993619 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:42:39.993619 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 10 23:42:39.993619 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:39.993619 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 337/685] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:42:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:39.993619 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56630a90-8e72-4f71-b1b4-8dea24b490c7 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:39.999905 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebbb10-6d09-4e54-a861-460a1e3ac6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:40.000452 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:40.000560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebbb10-6d09-4e54-a861-460a1e3ac6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:40.001039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebbb10-6d09-4e54-a861-460a1e3ac6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:40.002654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebbb10-6d09-4e54-a861-460a1e3ac6b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:40.002654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ebbb10-6d09-4e54-a861-460a1e3ac6b0 None admin] RBAC: Adding query filter params (role.id=8ff1727b65104e5fb97333d477196d46, scope.project.id=947277c4458645899287e6818947c076, user.id=53e79d9fb2cd452d8d999a84b6425959) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:40.002654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ebbb10-6d09-4e54-a861-460a1e3ac6b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:40.004198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ebbb10-6d09-4e54-a861-460a1e3ac6b0 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:40.011412 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 349/686] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:42:39 2022] GET /identity/v3/role_assignments?user.id=53e79d9fb2cd452d8d999a84b6425959&scope.project.id=947277c4458645899287e6818947c076&role.id=8ff1727b65104e5fb97333d477196d46 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:40.012844 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebbb10-6d09-4e54-a861-460a1e3ac6b0 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:40.151296 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56630a90-8e72-4f71-b1b4-8dea24b490c7 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:40.163204 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebbb10-6d09-4e54-a861-460a1e3ac6b0 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:40.195961 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d49db-b827-4e5e-a85c-6a73d28333a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:40.196673 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d49db-b827-4e5e-a85c-6a73d28333a7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:40.197047 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d49db-b827-4e5e-a85c-6a73d28333a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:40.197414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d49db-b827-4e5e-a85c-6a73d28333a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:40.198441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d49db-b827-4e5e-a85c-6a73d28333a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:40.198563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d49db-b827-4e5e-a85c-6a73d28333a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:40.200994 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d49db-b827-4e5e-a85c-6a73d28333a7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:40.211369 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5beba-dd8f-40a1-9834-3e3ec335565d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:40.212208 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5beba-dd8f-40a1-9834-3e3ec335565d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:40.212416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 338/687] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:42:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:40.212796 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5beba-dd8f-40a1-9834-3e3ec335565d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:40.214011 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5beba-dd8f-40a1-9834-3e3ec335565d None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:40.214956 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5beba-dd8f-40a1-9834-3e3ec335565d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:40.215394 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5beba-dd8f-40a1-9834-3e3ec335565d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:40.217036 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5beba-dd8f-40a1-9834-3e3ec335565d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:40.218072 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d49db-b827-4e5e-a85c-6a73d28333a7 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:40.240434 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 350/688] 10.222.0.17 () {56 vars in 1135 bytes} [Sat Sep 10 23:42:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:40.246106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5beba-dd8f-40a1-9834-3e3ec335565d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:40.262314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:40.310031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:40.354035 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d49db-b827-4e5e-a85c-6a73d28333a7 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:40.383247 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5beba-dd8f-40a1-9834-3e3ec335565d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:40.395990 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3b8ae-f497-408a-abb5-230c4866688f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:40.396637 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3b8ae-f497-408a-abb5-230c4866688f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:40.396953 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3b8ae-f497-408a-abb5-230c4866688f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:40.397224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3b8ae-f497-408a-abb5-230c4866688f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:40.409748 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3b8ae-f497-408a-abb5-230c4866688f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:40.412807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3b8ae-f497-408a-abb5-230c4866688f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:40.424223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46da928-d3fb-4ba3-a390-a89240c65fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:40.424968 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46da928-d3fb-4ba3-a390-a89240c65fab None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:40.425339 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46da928-d3fb-4ba3-a390-a89240c65fab None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:40.425699 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46da928-d3fb-4ba3-a390-a89240c65fab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:40.430567 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-42b3b8ae-f497-408a-abb5-230c4866688f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:40.430567 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:40.430567 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:40.430567 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:40.430567 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:40.430567 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:40.430567 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:40.430567 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:40.430567 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:40.430567 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:40.430567 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:40.430567 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:40.430567 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:40.430567 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:40.430567 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:40.430567 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:40.431522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:40.431522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:40.431522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:40.431522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:40.431522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:40.431522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:40.431522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:40.431522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:40.431522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:40.431522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:40.431522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:40.431522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:40.431522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:40.431522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:40.431522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:40.431522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:40.431522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:40.432727 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:40.432727 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:40.432727 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:40.432727 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 339/689] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:40.438026 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3b8ae-f497-408a-abb5-230c4866688f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:40.561593 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46da928-d3fb-4ba3-a390-a89240c65fab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:40.563089 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46da928-d3fb-4ba3-a390-a89240c65fab None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:40.595014 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:40.613599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3b8ae-f497-408a-abb5-230c4866688f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:40.632400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:40.654878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d10c52-491d-4325-8a6c-26cd18e9db6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:40.655492 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d10c52-491d-4325-8a6c-26cd18e9db6e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:40.655768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d10c52-491d-4325-8a6c-26cd18e9db6e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:40.656037 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d10c52-491d-4325-8a6c-26cd18e9db6e None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:40.656678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d10c52-491d-4325-8a6c-26cd18e9db6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:40.656977 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d10c52-491d-4325-8a6c-26cd18e9db6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:40.658344 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d10c52-491d-4325-8a6c-26cd18e9db6e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:40.697447 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 340/690] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:40.702611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d10c52-491d-4325-8a6c-26cd18e9db6e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:40.738337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 351/691] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:42:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:42:40.747462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46da928-d3fb-4ba3-a390-a89240c65fab None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:40.849426 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d10c52-491d-4325-8a6c-26cd18e9db6e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:40.903928 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46da928-d3fb-4ba3-a390-a89240c65fab None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:40.917410 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:40.930842 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2afeb8-fec8-4562-843a-77f07a60ebb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:40.931480 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2afeb8-fec8-4562-843a-77f07a60ebb7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:40.931754 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2afeb8-fec8-4562-843a-77f07a60ebb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:40.932282 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2afeb8-fec8-4562-843a-77f07a60ebb7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:40.945436 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2afeb8-fec8-4562-843a-77f07a60ebb7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:40.947170 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2afeb8-fec8-4562-843a-77f07a60ebb7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:40.966358 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-ab2afeb8-fec8-4562-843a-77f07a60ebb7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 23:42:40.966358 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:40.966358 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:40.966358 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:40.966358 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:40.966358 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:40.966358 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:40.966358 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:40.966358 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:40.966358 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:40.966358 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:40.966358 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:40.966358 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:42:40.966358 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:42:40.966358 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:42:40.966358 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:42:40.967484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:40.967484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:40.967484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:42:40.967484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:40.967484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:42:40.967484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:40.967484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:40.967484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:40.967484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:40.967484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:40.967484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:40.967484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:40.967484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:40.967484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:40.967484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:40.967484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:40.967484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:40.967484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:40.968418 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:40.968418 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:40.968418 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:42:40.968418 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:42:40.968418 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:42:40.968418 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:42:40.968418 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:42:40.968418 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:42:40.968418 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 23:42:40.968418 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:40.968418 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 341/692] 10.222.0.17 () {56 vars in 1093 bytes} [Sat Sep 10 23:42:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:40.976571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d492723-4982-42c4-bdc0-3505b01df90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:40.977264 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d492723-4982-42c4-bdc0-3505b01df90e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:40.977544 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d492723-4982-42c4-bdc0-3505b01df90e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:40.977871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d492723-4982-42c4-bdc0-3505b01df90e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:40.983482 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2afeb8-fec8-4562-843a-77f07a60ebb7 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:40.990150 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d492723-4982-42c4-bdc0-3505b01df90e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:40.991737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d492723-4982-42c4-bdc0-3505b01df90e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:41.008441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-5d492723-4982-42c4-bdc0-3505b01df90e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:41.008441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:41.008441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:41.008441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:41.008441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:41.008441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:41.008441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:41.008441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:41.008441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:41.008441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:41.008441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:41.008441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:41.008441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:41.008441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:41.008441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:41.008441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:41.009457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:41.009457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:41.009457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:41.009457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:41.009457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:41.009457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:41.009457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:41.009457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:41.009457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:41.009457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:41.009457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:41.009457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:41.009457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:41.009457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:41.009457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:41.009457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:41.009457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:41.010355 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:41.010355 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:41.010355 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:41.010355 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 352/693] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:41.015950 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d492723-4982-42c4-bdc0-3505b01df90e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:41.064311 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:41.179539 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2afeb8-fec8-4562-843a-77f07a60ebb7 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:41.195350 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d492723-4982-42c4-bdc0-3505b01df90e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:41.247151 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52582209-0010-4a5e-aeda-da27f4ab274a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:41.247752 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52582209-0010-4a5e-aeda-da27f4ab274a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:41.248027 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52582209-0010-4a5e-aeda-da27f4ab274a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:41.248293 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52582209-0010-4a5e-aeda-da27f4ab274a None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:41.249021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52582209-0010-4a5e-aeda-da27f4ab274a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:41.249322 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52582209-0010-4a5e-aeda-da27f4ab274a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:41.251387 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52582209-0010-4a5e-aeda-da27f4ab274a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:41.261100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92725f49-4eff-442d-8ef3-84b8ce610c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:41.262007 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92725f49-4eff-442d-8ef3-84b8ce610c03 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:41.262374 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92725f49-4eff-442d-8ef3-84b8ce610c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:41.262648 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92725f49-4eff-442d-8ef3-84b8ce610c03 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:41.263314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92725f49-4eff-442d-8ef3-84b8ce610c03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:41.263614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92725f49-4eff-442d-8ef3-84b8ce610c03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:41.265085 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92725f49-4eff-442d-8ef3-84b8ce610c03 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:41.276266 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 342/694] 10.222.0.17 () {56 vars in 1133 bytes} [Sat Sep 10 23:42:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:41.283401 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52582209-0010-4a5e-aeda-da27f4ab274a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:41.290723 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 353/695] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:41.309828 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92725f49-4eff-442d-8ef3-84b8ce610c03 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:41.347416 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:41.484526 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92725f49-4eff-442d-8ef3-84b8ce610c03 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:41.491701 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:41.501066 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52582209-0010-4a5e-aeda-da27f4ab274a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:41.523441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3320d-1e82-4756-b921-ebe3eb4444dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:41.524157 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3320d-1e82-4756-b921-ebe3eb4444dc None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:41.524434 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3320d-1e82-4756-b921-ebe3eb4444dc None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:41.524713 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3320d-1e82-4756-b921-ebe3eb4444dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:41.531818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c3320d-1e82-4756-b921-ebe3eb4444dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:41.533379 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c3320d-1e82-4756-b921-ebe3eb4444dc None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:41.543807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5f690-36d3-4ba6-9fbe-e689ee2f32d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:41.544650 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5f690-36d3-4ba6-9fbe-e689ee2f32d2 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:41.545161 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5f690-36d3-4ba6-9fbe-e689ee2f32d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:41.545642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5f690-36d3-4ba6-9fbe-e689ee2f32d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:41.545755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-f6c3320d-1e82-4756-b921-ebe3eb4444dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:41.545755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:41.545755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:41.545755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:41.545755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:41.545755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:41.545755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:41.545755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:41.545755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:41.545755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:41.545755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:41.545755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:41.545755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:42:41.545755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:42:41.545755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:42:41.546564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:42:41.546564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:41.546564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:41.546564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:41.546564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:41.546564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:41.546564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:41.546564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:41.546564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:41.546564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:41.546564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:41.546564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:41.546564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:41.546564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:41.546564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:41.546564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:41.546564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:41.547537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:42:41.547537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:42:41.547537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:42:41.547537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:42:41.547537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:42:41.547537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:42:41.547537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:41.547537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:41.547537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 354/696] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:41.548449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e65d6-ed91-4108-bd07-8889d0dbd5a4 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:41.548544 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e65d6-ed91-4108-bd07-8889d0dbd5a4 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:41.548770 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e65d6-ed91-4108-bd07-8889d0dbd5a4 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:41.554107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 355/697] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:41.554107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e65d6-ed91-4108-bd07-8889d0dbd5a4 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:41.638757 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5f690-36d3-4ba6-9fbe-e689ee2f32d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:41.640874 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5f690-36d3-4ba6-9fbe-e689ee2f32d2 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:41.645572 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:41.657085 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e65d6-ed91-4108-bd07-8889d0dbd5a4 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:41.695589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be1515-95df-4f24-848a-1e25465e1027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:41.696207 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be1515-95df-4f24-848a-1e25465e1027 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:41.696471 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be1515-95df-4f24-848a-1e25465e1027 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:41.696744 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be1515-95df-4f24-848a-1e25465e1027 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:41.697429 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be1515-95df-4f24-848a-1e25465e1027 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:41.697733 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be1515-95df-4f24-848a-1e25465e1027 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:41.699747 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be1515-95df-4f24-848a-1e25465e1027 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:41.712043 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 356/698] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:42:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:41.713293 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742266d5-6259-45fd-ab37-3740865cec64 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:41.713579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742266d5-6259-45fd-ab37-3740865cec64 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:41.713852 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742266d5-6259-45fd-ab37-3740865cec64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:41.738689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-742266d5-6259-45fd-ab37-3740865cec64 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:41.739794 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-742266d5-6259-45fd-ab37-3740865cec64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:41.767313 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:41.783352 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 343/699] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:42:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 10 23:42:41.784477 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5f690-36d3-4ba6-9fbe-e689ee2f32d2 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:41.815001 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 357/700] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:41.815694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742266d5-6259-45fd-ab37-3740865cec64 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:41.891724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:41.923932 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742266d5-6259-45fd-ab37-3740865cec64 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:41.961814 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b46c21-a7ed-4abe-8b04-1ac5ef7e4c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:41.962443 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b46c21-a7ed-4abe-8b04-1ac5ef7e4c19 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:41.962751 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b46c21-a7ed-4abe-8b04-1ac5ef7e4c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:41.963060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b46c21-a7ed-4abe-8b04-1ac5ef7e4c19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:41.974197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b46c21-a7ed-4abe-8b04-1ac5ef7e4c19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:41.975696 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b46c21-a7ed-4abe-8b04-1ac5ef7e4c19 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:41.991352 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5f690-36d3-4ba6-9fbe-e689ee2f32d2 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:41.993649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-06b46c21-a7ed-4abe-8b04-1ac5ef7e4c19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:41.993649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:41.993649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:41.993649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:41.993649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:41.993649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:41.993649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:41.993649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:41.993649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:41.993649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:41.993649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:41.993649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:41.993649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:41.993649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:41.993649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:41.993649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:41.994641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:41.994641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:41.994641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:41.994641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:41.994641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:41.994641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:41.994641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:41.994641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:41.994641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:41.994641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:41.994641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:41.994641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:41.994641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:41.994641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:41.994641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:41.994641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:41.994641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:41.995589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:41.995589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:41.995589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:41.995589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 358/701] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:42.002119 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b46c21-a7ed-4abe-8b04-1ac5ef7e4c19 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:42.038310 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87267c-36d2-4b9f-9a23-c2b82ca95d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:42.038978 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87267c-36d2-4b9f-9a23-c2b82ca95d53 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:42.039257 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87267c-36d2-4b9f-9a23-c2b82ca95d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:42.039520 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87267c-36d2-4b9f-9a23-c2b82ca95d53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:42.040245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab87267c-36d2-4b9f-9a23-c2b82ca95d53 None admin] RBAC: Adding query filter params (role.id=8ff1727b65104e5fb97333d477196d46, scope.project.id=947277c4458645899287e6818947c076, user.id=564e65585e9543e286b6041af870cf41) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:42.040543 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab87267c-36d2-4b9f-9a23-c2b82ca95d53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:42.041973 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab87267c-36d2-4b9f-9a23-c2b82ca95d53 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:42.056061 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 344/702] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:42:41 2022] GET /identity/v3/role_assignments?user.id=564e65585e9543e286b6041af870cf41&scope.project.id=947277c4458645899287e6818947c076&role.id=8ff1727b65104e5fb97333d477196d46 => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:42.142534 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87267c-36d2-4b9f-9a23-c2b82ca95d53 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:42.177003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b46c21-a7ed-4abe-8b04-1ac5ef7e4c19 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:42.225279 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:42.249616 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc8c82-4e46-48ac-90ec-015fe7a81f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:42.250149 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc8c82-4e46-48ac-90ec-015fe7a81f5f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:42.250418 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc8c82-4e46-48ac-90ec-015fe7a81f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:42.250697 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc8c82-4e46-48ac-90ec-015fe7a81f5f None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:42.251422 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc8c82-4e46-48ac-90ec-015fe7a81f5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:42.251719 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc8c82-4e46-48ac-90ec-015fe7a81f5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:42.253153 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc8c82-4e46-48ac-90ec-015fe7a81f5f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:42.255152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:42.280645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 359/703] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:42.286321 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc8c82-4e46-48ac-90ec-015fe7a81f5f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:42.331899 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87267c-36d2-4b9f-9a23-c2b82ca95d53 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:42.382931 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123f09e-9206-44fb-83b9-d08010760db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:42.384007 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123f09e-9206-44fb-83b9-d08010760db2 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:42.384554 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123f09e-9206-44fb-83b9-d08010760db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:42.385023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123f09e-9206-44fb-83b9-d08010760db2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:42.385865 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123f09e-9206-44fb-83b9-d08010760db2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:42.388885 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123f09e-9206-44fb-83b9-d08010760db2 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:42.407638 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-0123f09e-9206-44fb-83b9-d08010760db2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 23:42:42.407638 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:42.407638 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 23:42:42.407638 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 23:42:42.407638 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 23:42:42.407638 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 23:42:42.407638 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 23:42:42.407638 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:42:42.407638 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 23:42:42.407638 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:42.407638 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:42:42.407638 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:42.407638 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:42.407638 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:42.407638 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:42.407638 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:42.407638 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:42.408729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:42.408729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:42.408729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:42.408729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:42.408729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:42.408729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:42.408729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 23:42:42.408729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 23:42:42.408729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 23:42:42.408729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 23:42:42.408729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:42.408729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:42.408729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:42.408729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:42.408729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:42.408729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:42.409604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:42.409604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:42.409604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:42.409604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:42.409604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:42.409604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:42.409604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:42.409604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:42.409604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:42.409604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:42.409604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 23:42:42.409604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:42:42.409604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 23:42:42.409604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:42.412400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 345/704] 10.222.0.17 () {56 vars in 1129 bytes} [Sat Sep 10 23:42:42 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:42:42.419821 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123f09e-9206-44fb-83b9-d08010760db2 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:42.432708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc8c82-4e46-48ac-90ec-015fe7a81f5f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:42.477663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a2c87-4212-4373-9465-2a99625e94b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:42.478639 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a2c87-4212-4373-9465-2a99625e94b3 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:42.479145 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a2c87-4212-4373-9465-2a99625e94b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:42.479645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a2c87-4212-4373-9465-2a99625e94b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:42.490074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73a2c87-4212-4373-9465-2a99625e94b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:42.492249 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73a2c87-4212-4373-9465-2a99625e94b3 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:42.506305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-a73a2c87-4212-4373-9465-2a99625e94b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:42.506305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:42.506305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:42.506305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:42.506305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:42.506305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:42.506305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:42.506305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:42.506305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:42.506305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:42.506305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:42.506305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:42.506305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:42:42.506305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:42:42.506305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:42:42.507851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:42:42.507851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:42.507851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:42.507851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:42.507851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:42.507851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:42.507851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:42.507851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:42.507851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:42.507851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:42.507851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:42.507851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:42.507851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:42.507851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:42.507851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:42.507851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:42.507851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:42.508756 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:42:42.508756 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:42:42.508756 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:42:42.508756 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:42:42.508756 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:42:42.508756 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:42:42.508756 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:42.508756 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:42.511204 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 360/705] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:42:42.517672 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7be27c-c02c-4ff1-a277-e2d2557b1828 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:42.517953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7be27c-c02c-4ff1-a277-e2d2557b1828 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:42.518225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7be27c-c02c-4ff1-a277-e2d2557b1828 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:42.520224 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 361/706] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:42:42.524138 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7be27c-c02c-4ff1-a277-e2d2557b1828 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:42.537547 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123f09e-9206-44fb-83b9-d08010760db2 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:42.568804 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:42.581078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d14d5-dea7-459d-aee0-350c99d17390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:42.581680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d14d5-dea7-459d-aee0-350c99d17390 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:42.581954 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d14d5-dea7-459d-aee0-350c99d17390 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:42.582227 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d14d5-dea7-459d-aee0-350c99d17390 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:42.582894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d14d5-dea7-459d-aee0-350c99d17390 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:42.583194 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d14d5-dea7-459d-aee0-350c99d17390 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:42.584579 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d14d5-dea7-459d-aee0-350c99d17390 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:42.586339 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:42.590163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 346/707] 10.222.0.17 () {56 vars in 1124 bytes} [Sat Sep 10 23:42:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:42.591409 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0d663-13aa-4481-a2ab-f30517019c9d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:42.591665 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0d663-13aa-4481-a2ab-f30517019c9d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:42.591933 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0d663-13aa-4481-a2ab-f30517019c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:42.623080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-85f0d663-13aa-4481-a2ab-f30517019c9d None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:42.624464 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-85f0d663-13aa-4481-a2ab-f30517019c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:42.658307 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7be27c-c02c-4ff1-a277-e2d2557b1828 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:42.703910 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e1ae0-1980-4486-82be-04d617b67d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:42.704883 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e1ae0-1980-4486-82be-04d617b67d8a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:42.705358 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e1ae0-1980-4486-82be-04d617b67d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:42.705853 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e1ae0-1980-4486-82be-04d617b67d8a None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:42.706967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e1ae0-1980-4486-82be-04d617b67d8a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:42.707478 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e1ae0-1980-4486-82be-04d617b67d8a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:42.709089 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e1ae0-1980-4486-82be-04d617b67d8a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:42.722706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 347/708] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:42.723031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 362/709] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:42:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:42.723654 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0d663-13aa-4481-a2ab-f30517019c9d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:42.724522 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5ef16-8de8-4b6c-bcea-b69760bbe0de None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:42.724803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5ef16-8de8-4b6c-bcea-b69760bbe0de None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:42.725070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5ef16-8de8-4b6c-bcea-b69760bbe0de None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:42.726493 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 363/710] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:42.729966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5ef16-8de8-4b6c-bcea-b69760bbe0de None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:42.842836 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0d663-13aa-4481-a2ab-f30517019c9d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:42.854808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5ef16-8de8-4b6c-bcea-b69760bbe0de None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:42.856689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:42.861323 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:42.881125 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f1549-9ac0-42b4-8e5d-301c0799e357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:42.882069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f1549-9ac0-42b4-8e5d-301c0799e357 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:42.882576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f1549-9ac0-42b4-8e5d-301c0799e357 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:42.883081 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f1549-9ac0-42b4-8e5d-301c0799e357 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:42.883907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f1549-9ac0-42b4-8e5d-301c0799e357 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:42.884409 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f1549-9ac0-42b4-8e5d-301c0799e357 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:42.886079 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77f1549-9ac0-42b4-8e5d-301c0799e357 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:42.892644 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 348/711] 10.222.0.17 () {56 vars in 1124 bytes} [Sat Sep 10 23:42:42 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:42.894213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabd01b-8d8e-4434-b1a0-7f08e3fe4d75 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:42.894784 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabd01b-8d8e-4434-b1a0-7f08e3fe4d75 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:42.895278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabd01b-8d8e-4434-b1a0-7f08e3fe4d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:42.901357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609af60f-d925-491b-9578-0fe8fc5872a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:42.903643 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609af60f-d925-491b-9578-0fe8fc5872a4 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:42.903643 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609af60f-d925-491b-9578-0fe8fc5872a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:42.903643 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609af60f-d925-491b-9578-0fe8fc5872a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:42.903865 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609af60f-d925-491b-9578-0fe8fc5872a4 None admin] RBAC: Adding query filter params (role.id=9dce48a2b53c482d931dcd064b07744e, scope.project.id=947277c4458645899287e6818947c076, user.id=636bfafb693044d283530063047fc0c2) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:42.904083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609af60f-d925-491b-9578-0fe8fc5872a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:42.905752 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609af60f-d925-491b-9578-0fe8fc5872a4 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:42.917217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-3eabd01b-8d8e-4434-b1a0-7f08e3fe4d75 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:42.918016 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 364/712] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:42:42 2022] GET /identity/v3/role_assignments?user.id=636bfafb693044d283530063047fc0c2&scope.project.id=947277c4458645899287e6818947c076&role.id=9dce48a2b53c482d931dcd064b07744e => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:42.923894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3eabd01b-8d8e-4434-b1a0-7f08e3fe4d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:43.014687 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609af60f-d925-491b-9578-0fe8fc5872a4 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:43.041267 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 349/713] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:43.156910 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609af60f-d925-491b-9578-0fe8fc5872a4 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:43.198697 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d7289-3d3a-42b8-8ff3-b15b7d452cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:43.200115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d7289-3d3a-42b8-8ff3-b15b7d452cfb None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:43.200390 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d7289-3d3a-42b8-8ff3-b15b7d452cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:43.200655 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d7289-3d3a-42b8-8ff3-b15b7d452cfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:43.214690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793d7289-3d3a-42b8-8ff3-b15b7d452cfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:43.216174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793d7289-3d3a-42b8-8ff3-b15b7d452cfb None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:43.233337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-793d7289-3d3a-42b8-8ff3-b15b7d452cfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:43.233337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:43.233337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:43.233337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:43.233337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:43.233337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:43.233337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:43.233337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:43.233337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:43.233337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:43.233337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:43.233337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:43.233337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:43.233337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:43.233337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:43.233337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:43.234871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:43.234871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:43.234871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:43.234871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:43.234871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:43.234871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:43.234871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:43.234871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:43.234871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:43.234871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:43.234871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:43.234871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:43.234871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:43.234871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:43.234871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:43.234871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:43.234871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:43.235796 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:43.235796 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:43.235796 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:43.238791 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 365/714] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:42:43.243244 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabd01b-8d8e-4434-b1a0-7f08e3fe4d75 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:43.371222 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d7289-3d3a-42b8-8ff3-b15b7d452cfb None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:43.424397 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:43.538139 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d7289-3d3a-42b8-8ff3-b15b7d452cfb None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:43.564706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabd01b-8d8e-4434-b1a0-7f08e3fe4d75 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:43.574311 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5011f-4ebd-4dac-8ca9-eb153e512c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:43.574965 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5011f-4ebd-4dac-8ca9-eb153e512c3a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:43.575240 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5011f-4ebd-4dac-8ca9-eb153e512c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:43.575508 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5011f-4ebd-4dac-8ca9-eb153e512c3a None admin] PATH_INFO: `/v3/services/network` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:43.576195 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db5011f-4ebd-4dac-8ca9-eb153e512c3a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:43.577555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db5011f-4ebd-4dac-8ca9-eb153e512c3a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:43.586424 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:43.599359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-9db5011f-4ebd-4dac-8ca9-eb153e512c3a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 23:42:43.599359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:43.599359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 23:42:43.599359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 23:42:43.599359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 23:42:43.599359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 23:42:43.599359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 23:42:43.599359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:42:43.599359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 23:42:43.599359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:43.599359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:42:43.599359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:43.599359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:43.599359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:43.599359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:43.599359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:43.599359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:43.604014 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:43.604014 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:43.604014 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:43.604014 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:43.604014 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:43.604014 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:43.604014 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 23:42:43.604014 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 23:42:43.604014 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 23:42:43.604014 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 23:42:43.604014 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:43.604014 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:43.604014 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:43.604014 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:43.604014 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:43.604014 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:43.604862 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:43.604862 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:43.604862 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:43.604862 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:43.604862 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:43.604862 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:43.604862 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:43.604862 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:43.604862 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:43.604862 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:43.604862 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 23:42:43.604862 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:42:43.604862 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 23:42:43.604862 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:43.604862 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 366/715] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:43 2022] GET /identity/v3/services/network => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:42:43.611668 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5011f-4ebd-4dac-8ca9-eb153e512c3a None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:43.635375 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b96d58-c2bd-432c-bd7d-15fb95e5bbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:43.636337 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b96d58-c2bd-432c-bd7d-15fb95e5bbd0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:43.636770 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b96d58-c2bd-432c-bd7d-15fb95e5bbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:43.637173 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b96d58-c2bd-432c-bd7d-15fb95e5bbd0 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:43.637984 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b96d58-c2bd-432c-bd7d-15fb95e5bbd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:43.638429 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b96d58-c2bd-432c-bd7d-15fb95e5bbd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:43.639981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b96d58-c2bd-432c-bd7d-15fb95e5bbd0 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:43.671094 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 350/716] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:43.698744 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b96d58-c2bd-432c-bd7d-15fb95e5bbd0 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:43.715593 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:43.736008 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5011f-4ebd-4dac-8ca9-eb153e512c3a None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:43.775624 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545efaf-3aa4-4a5c-a479-d69a7304a795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:43.776384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545efaf-3aa4-4a5c-a479-d69a7304a795 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:43.777781 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545efaf-3aa4-4a5c-a479-d69a7304a795 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:43.778198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545efaf-3aa4-4a5c-a479-d69a7304a795 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:43.779107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4545efaf-3aa4-4a5c-a479-d69a7304a795 None admin] RBAC: Adding query filter params (name=network) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:43.779642 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4545efaf-3aa4-4a5c-a479-d69a7304a795 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:43.781132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4545efaf-3aa4-4a5c-a479-d69a7304a795 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:43.786409 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 367/717] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Sep 10 23:42:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:43.792616 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545efaf-3aa4-4a5c-a479-d69a7304a795 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:43.910685 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545efaf-3aa4-4a5c-a479-d69a7304a795 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:43.945987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1ecf5-2494-4533-8b95-a1fa2cb3bcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:43.946882 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1ecf5-2494-4533-8b95-a1fa2cb3bcde None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:43.947273 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1ecf5-2494-4533-8b95-a1fa2cb3bcde None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:43.947637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1ecf5-2494-4533-8b95-a1fa2cb3bcde None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:43.948462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1ecf5-2494-4533-8b95-a1fa2cb3bcde None admin] RBAC: Adding query filter params (type=network) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:43.949481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1ecf5-2494-4533-8b95-a1fa2cb3bcde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:43.954684 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1ecf5-2494-4533-8b95-a1fa2cb3bcde None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:43.957671 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 368/718] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Sep 10 23:42:43 2022] GET /identity/v3/services?type=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:43.999242 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b96d58-c2bd-432c-bd7d-15fb95e5bbd0 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:44.061948 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45339a96-d88a-44c7-b42f-b7e6d8e84a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:44.062857 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45339a96-d88a-44c7-b42f-b7e6d8e84a61 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:44.063283 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45339a96-d88a-44c7-b42f-b7e6d8e84a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:44.063697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45339a96-d88a-44c7-b42f-b7e6d8e84a61 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:44.064550 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45339a96-d88a-44c7-b42f-b7e6d8e84a61 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:44.066166 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45339a96-d88a-44c7-b42f-b7e6d8e84a61 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:44.164515 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 351/719] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Sep 10 23:42:43 2022] POST /identity/v3/users => generated 254 bytes in 473 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 23:42:44.771626 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436391b8-0fad-4cab-a492-8a45b870a882 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:44.772150 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436391b8-0fad-4cab-a492-8a45b870a882 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:44.772150 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436391b8-0fad-4cab-a492-8a45b870a882 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:44.773917 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 369/720] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:44.779339 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7c38d-ac51-4029-843f-39310cc83605 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:44.779766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7c38d-ac51-4029-843f-39310cc83605 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:44.779941 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7c38d-ac51-4029-843f-39310cc83605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:44.801528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-45f7c38d-ac51-4029-843f-39310cc83605 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:44.802765 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-45f7c38d-ac51-4029-843f-39310cc83605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:44.830984 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:44.889779 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 352/721] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:45.250978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436391b8-0fad-4cab-a492-8a45b870a882 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:45.264850 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:45.381728 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436391b8-0fad-4cab-a492-8a45b870a882 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:45.416528 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c249ea-17e3-4ade-89bd-68bb831a08e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:45.417165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c249ea-17e3-4ade-89bd-68bb831a08e3 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:45.417982 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c249ea-17e3-4ade-89bd-68bb831a08e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:45.418257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c249ea-17e3-4ade-89bd-68bb831a08e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:45.429427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c249ea-17e3-4ade-89bd-68bb831a08e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:45.434691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-a4c249ea-17e3-4ade-89bd-68bb831a08e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:42:45.434691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:45.434691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:45.434691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:45.434691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:45.434691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:45.434691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:45.434691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:45.434691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:45.434691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:45.434691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:45.434691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:45.434691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:45.434691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:45.434691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:45.434691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:45.435625 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:45.435625 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:45.435625 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:45.435625 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:45.435625 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:45.435625 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:45.435625 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:45.435625 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:45.435625 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:45.435625 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:45.435625 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:45.435625 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:45.435625 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:45.435625 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:45.435625 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:45.435625 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:45.436462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:45.436462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:45.436462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:45.436462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:45.436462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:45.436462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:45.436462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:45.436462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:45.436462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:45.436462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:45.436462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:42:45.436462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:45.436462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 370/722] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:42:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:45.441922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7c38d-ac51-4029-843f-39310cc83605 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:45.531772 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132de018-5e3d-4786-9b92-b955367d0ca0 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:45.532409 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132de018-5e3d-4786-9b92-b955367d0ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:45.532913 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132de018-5e3d-4786-9b92-b955367d0ca0 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:45.534763 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 371/723] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:42:45.539930 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914bd9f-8a48-4fae-8d23-af21ff5d16e2 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:45.540511 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914bd9f-8a48-4fae-8d23-af21ff5d16e2 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:45.540852 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914bd9f-8a48-4fae-8d23-af21ff5d16e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:45.567001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7c38d-ac51-4029-843f-39310cc83605 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:45.583836 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a914bd9f-8a48-4fae-8d23-af21ff5d16e2 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:45.585053 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a914bd9f-8a48-4fae-8d23-af21ff5d16e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:45.620032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb06c4a7-9457-491a-bd77-2f29d294026a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:45.620669 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06c4a7-9457-491a-bd77-2f29d294026a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:45.621035 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06c4a7-9457-491a-bd77-2f29d294026a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:45.621525 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06c4a7-9457-491a-bd77-2f29d294026a None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:45.622198 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb06c4a7-9457-491a-bd77-2f29d294026a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:45.622502 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb06c4a7-9457-491a-bd77-2f29d294026a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:45.623918 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb06c4a7-9457-491a-bd77-2f29d294026a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:45.634089 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 353/724] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:42:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:45.641356 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb06c4a7-9457-491a-bd77-2f29d294026a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:45.671328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:45.673233 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:45.682749 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 372/725] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:45.684455 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1779e-0460-4707-b7df-527a21fe0d0a None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:45.685015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1779e-0460-4707-b7df-527a21fe0d0a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:45.685497 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1779e-0460-4707-b7df-527a21fe0d0a None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:45.692706 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 373/726] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:45.692706 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb403e4a-e49b-4b71-a6e4-88681973ae22 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:45.693267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb403e4a-e49b-4b71-a6e4-88681973ae22 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:45.693729 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb403e4a-e49b-4b71-a6e4-88681973ae22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:45.717833 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-bb403e4a-e49b-4b71-a6e4-88681973ae22 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:45.720584 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-bb403e4a-e49b-4b71-a6e4-88681973ae22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:45.804845 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 374/727] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 23:42:45.836436 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb06c4a7-9457-491a-bd77-2f29d294026a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:45.888345 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0311e70-c916-407d-831f-e1872570d128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:45.888976 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0311e70-c916-407d-831f-e1872570d128 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:45.889250 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0311e70-c916-407d-831f-e1872570d128 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:45.889519 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0311e70-c916-407d-831f-e1872570d128 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:45.903778 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0311e70-c916-407d-831f-e1872570d128 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:45.905305 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0311e70-c916-407d-831f-e1872570d128 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:45.920904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-d0311e70-c916-407d-831f-e1872570d128 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:45.920904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:45.920904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:45.920904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:45.920904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:45.920904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:45.920904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:45.920904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:45.920904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:45.920904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:45.920904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:45.920904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:45.920904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:45.920904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:45.920904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:45.920904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:45.921880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:45.921880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:45.921880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:45.921880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:45.921880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:45.921880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:45.921880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:45.921880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:45.921880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:45.921880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:45.921880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:45.921880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:45.921880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:45.921880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:45.921880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:45.921880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:45.921880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:45.922811 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:45.922811 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:45.922811 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:45.922811 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 354/728] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:42:45.927966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb403e4a-e49b-4b71-a6e4-88681973ae22 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:46.056891 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb403e4a-e49b-4b71-a6e4-88681973ae22 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:46.082540 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0311e70-c916-407d-831f-e1872570d128 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:46.088385 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:46.095671 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63710923-7ce9-49e1-b468-49478655d00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:46.096311 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63710923-7ce9-49e1-b468-49478655d00d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:46.096594 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63710923-7ce9-49e1-b468-49478655d00d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:46.096865 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63710923-7ce9-49e1-b468-49478655d00d None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:46.097511 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63710923-7ce9-49e1-b468-49478655d00d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:46.097810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63710923-7ce9-49e1-b468-49478655d00d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:46.099221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63710923-7ce9-49e1-b468-49478655d00d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:46.111341 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 375/729] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:46.117207 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63710923-7ce9-49e1-b468-49478655d00d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:46.194163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:46.237820 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0311e70-c916-407d-831f-e1872570d128 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:46.238707 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63710923-7ce9-49e1-b468-49478655d00d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:46.273722 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35039c58-5eaf-468c-9031-fcea14ca5a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:46.274324 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35039c58-5eaf-468c-9031-fcea14ca5a66 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:46.274600 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35039c58-5eaf-468c-9031-fcea14ca5a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:46.274892 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35039c58-5eaf-468c-9031-fcea14ca5a66 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:46.275598 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35039c58-5eaf-468c-9031-fcea14ca5a66 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:46.277053 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35039c58-5eaf-468c-9031-fcea14ca5a66 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:46.282417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3976ac4-deee-4d38-beef-0bc5b26aaf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:46.283105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3976ac4-deee-4d38-beef-0bc5b26aaf8e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:46.283991 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3976ac4-deee-4d38-beef-0bc5b26aaf8e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:46.284394 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3976ac4-deee-4d38-beef-0bc5b26aaf8e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:46.292869 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3976ac4-deee-4d38-beef-0bc5b26aaf8e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:46.295147 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3976ac4-deee-4d38-beef-0bc5b26aaf8e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:46.316735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-e3976ac4-deee-4d38-beef-0bc5b26aaf8e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 10 23:42:46.316735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:46.316735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:46.316735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:46.316735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:46.316735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:46.316735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:46.316735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:46.316735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:46.316735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:46.316735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:46.316735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:46.316735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:42:46.316735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:42:46.316735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:42:46.316735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:42:46.317664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:46.317664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:46.317664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:42:46.317664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:46.317664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:42:46.317664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:46.317664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:46.317664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:46.317664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:46.317664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:46.317664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:46.317664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:46.317664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:46.317664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:46.317664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:46.317664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:46.317664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:46.317664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:46.318598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:46.318598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:46.318598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:42:46.318598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:42:46.318598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:42:46.318598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:42:46.318598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:42:46.318598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:42:46.318598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 10 23:42:46.318598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:46.318598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 376/730] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:42:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:46.318598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3976ac4-deee-4d38-beef-0bc5b26aaf8e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:46.354372 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 355/731] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Sep 10 23:42:46 2022] POST /identity/v3/services => generated 251 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:42:46.355651 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35039c58-5eaf-468c-9031-fcea14ca5a66 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:46.385380 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:46.461156 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3976ac4-deee-4d38-beef-0bc5b26aaf8e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:46.486712 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35039c58-5eaf-468c-9031-fcea14ca5a66 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:46.506502 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f581a-4f49-4ebc-9ece-08d4b55149f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:46.507694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f581a-4f49-4ebc-9ece-08d4b55149f0 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:46.508510 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f581a-4f49-4ebc-9ece-08d4b55149f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:46.508970 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f581a-4f49-4ebc-9ece-08d4b55149f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:46.518529 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:46.523003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818f581a-4f49-4ebc-9ece-08d4b55149f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:46.529415 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99c8fa4-ba33-4811-aa75-1ed48cf8be44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:46.530140 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c8fa4-ba33-4811-aa75-1ed48cf8be44 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:46.530476 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c8fa4-ba33-4811-aa75-1ed48cf8be44 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:46.530809 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c8fa4-ba33-4811-aa75-1ed48cf8be44 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:46.531198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-818f581a-4f49-4ebc-9ece-08d4b55149f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:42:46.531198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:46.531198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:46.531198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:46.531198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:46.531198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:46.531198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:46.531198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:46.531198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:46.531198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:46.531198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:46.531198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:46.531198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:46.531198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:46.531198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:46.531198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:46.532102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:46.532102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:46.532102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:46.532102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:46.532102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:46.532102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:46.532102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:46.532102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:46.532102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:46.532102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:46.532102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:46.532102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:46.532102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:46.532102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:46.532102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:46.532102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:46.532950 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:46.532950 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:46.532950 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:46.532950 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:46.532950 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:46.532950 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:46.532950 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:46.532950 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:46.532950 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:46.532950 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:46.532950 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:42:46.532950 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:46.533575 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99c8fa4-ba33-4811-aa75-1ed48cf8be44 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:46.533575 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99c8fa4-ba33-4811-aa75-1ed48cf8be44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:46.533575 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99c8fa4-ba33-4811-aa75-1ed48cf8be44 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:46.542079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 377/732] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:42:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:46.544570 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be3688-9897-45f6-9a01-760ae29ec7f8 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:46.544844 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be3688-9897-45f6-9a01-760ae29ec7f8 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:46.545111 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be3688-9897-45f6-9a01-760ae29ec7f8 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:46.549975 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 378/733] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:46.550245 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be3688-9897-45f6-9a01-760ae29ec7f8 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:46.560740 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 356/734] 10.222.0.17 () {56 vars in 1135 bytes} [Sat Sep 10 23:42:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:46.567005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95751e27-c8de-4ea1-a0d4-8a0c8d983fb8 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:46.567284 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95751e27-c8de-4ea1-a0d4-8a0c8d983fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:46.567471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95751e27-c8de-4ea1-a0d4-8a0c8d983fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:46.588464 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-95751e27-c8de-4ea1-a0d4-8a0c8d983fb8 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:46.590138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-95751e27-c8de-4ea1-a0d4-8a0c8d983fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:46.678693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 357/735] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:46 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 23:42:46.678693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95751e27-c8de-4ea1-a0d4-8a0c8d983fb8 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:46.719376 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be3688-9897-45f6-9a01-760ae29ec7f8 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:46.740808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:46.766606 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ac58b-da16-4685-a1bd-1fdcbdd6dc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:46.767307 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ac58b-da16-4685-a1bd-1fdcbdd6dc31 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:46.767582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ac58b-da16-4685-a1bd-1fdcbdd6dc31 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:46.767850 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ac58b-da16-4685-a1bd-1fdcbdd6dc31 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:46.768509 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ac58b-da16-4685-a1bd-1fdcbdd6dc31 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:46.768806 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ac58b-da16-4685-a1bd-1fdcbdd6dc31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:46.770181 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ac58b-da16-4685-a1bd-1fdcbdd6dc31 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:46.785101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 379/736] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:42:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:46.785973 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ac58b-da16-4685-a1bd-1fdcbdd6dc31 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:46.892015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ac58b-da16-4685-a1bd-1fdcbdd6dc31 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:46.896588 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95751e27-c8de-4ea1-a0d4-8a0c8d983fb8 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:46.912386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:46.927324 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91edb08c-3502-473c-ba28-5a538dfaa4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:46.928007 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91edb08c-3502-473c-ba28-5a538dfaa4c0 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:46.928284 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91edb08c-3502-473c-ba28-5a538dfaa4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:46.928552 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91edb08c-3502-473c-ba28-5a538dfaa4c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:46.939980 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91edb08c-3502-473c-ba28-5a538dfaa4c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:46.941439 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91edb08c-3502-473c-ba28-5a538dfaa4c0 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:46.957319 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-91edb08c-3502-473c-ba28-5a538dfaa4c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:46.957319 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:46.957319 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:46.957319 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:46.957319 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:46.957319 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:46.957319 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:46.957319 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:46.957319 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:46.957319 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:46.957319 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:46.957319 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:46.957319 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:46.957319 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:46.957319 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:46.957319 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:46.958354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:46.958354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:46.958354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:46.958354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:46.958354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:46.958354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:46.958354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:46.958354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:46.958354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:46.958354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:46.958354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:46.958354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:46.958354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:46.958354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:46.958354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:46.958354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:46.958354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:46.959263 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:46.959263 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:46.959263 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:46.959263 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 380/737] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:46.964514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91edb08c-3502-473c-ba28-5a538dfaa4c0 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:46.979429 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4fd59-19b0-4c8b-b89a-c909145c8f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:46.980723 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4fd59-19b0-4c8b-b89a-c909145c8f6e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:46.981109 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4fd59-19b0-4c8b-b89a-c909145c8f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:46.981384 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4fd59-19b0-4c8b-b89a-c909145c8f6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:46.992416 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:47.081015 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c4fd59-19b0-4c8b-b89a-c909145c8f6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:47.081242 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91edb08c-3502-473c-ba28-5a538dfaa4c0 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:47.083269 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c4fd59-19b0-4c8b-b89a-c909145c8f6e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:47.118883 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787636a-f312-4864-b222-df3186f46dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:47.119578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787636a-f312-4864-b222-df3186f46dcd None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:47.120029 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787636a-f312-4864-b222-df3186f46dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:47.120370 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787636a-f312-4864-b222-df3186f46dcd None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:47.121314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787636a-f312-4864-b222-df3186f46dcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:47.121686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787636a-f312-4864-b222-df3186f46dcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:47.123254 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787636a-f312-4864-b222-df3186f46dcd None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:47.138549 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 381/738] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:47.139226 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787636a-f312-4864-b222-df3186f46dcd None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:47.226986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 358/739] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:42:46 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 23:42:47.229440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4fd59-19b0-4c8b-b89a-c909145c8f6e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:47.243938 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:47.246110 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:47.264205 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787636a-f312-4864-b222-df3186f46dcd None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:47.313490 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404123a7-120a-41a1-ab3d-c779f2b1a376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:47.314115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404123a7-120a-41a1-ab3d-c779f2b1a376 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:47.314387 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404123a7-120a-41a1-ab3d-c779f2b1a376 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:47.314653 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404123a7-120a-41a1-ab3d-c779f2b1a376 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:47.315333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404123a7-120a-41a1-ab3d-c779f2b1a376 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:47.316703 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404123a7-120a-41a1-ab3d-c779f2b1a376 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:47.352188 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4fd59-19b0-4c8b-b89a-c909145c8f6e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:47.394823 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 382/740] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Sep 10 23:42:47 2022] POST /identity/v3/services => generated 240 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:42:47.394823 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404123a7-120a-41a1-ab3d-c779f2b1a376 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:47.404004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc26eb-7a97-42ff-9c8c-63b787f006c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:47.406930 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc26eb-7a97-42ff-9c8c-63b787f006c2 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:47.407296 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc26eb-7a97-42ff-9c8c-63b787f006c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:47.407573 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc26eb-7a97-42ff-9c8c-63b787f006c2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:47.424865 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc26eb-7a97-42ff-9c8c-63b787f006c2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:47.426360 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc26eb-7a97-42ff-9c8c-63b787f006c2 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:47.452571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-32cc26eb-7a97-42ff-9c8c-63b787f006c2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 23:42:47.452571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:47.452571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:47.452571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:47.452571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:47.452571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:47.452571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:47.452571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:47.452571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:47.452571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:47.452571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:47.452571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:47.452571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:42:47.452571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:42:47.452571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:42:47.452571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:42:47.453590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:47.453590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:47.453590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:42:47.453590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:47.453590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:42:47.453590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:47.453590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:47.453590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:47.453590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:47.453590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:47.453590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:47.453590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:47.453590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:47.453590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:47.453590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:47.453590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:47.453590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:47.453590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:47.454533 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:47.454533 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:47.454533 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:42:47.454533 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:42:47.454533 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:42:47.454533 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:42:47.454533 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:42:47.454533 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:42:47.454533 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 23:42:47.454533 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:47.459326 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 359/741] 10.222.0.17 () {56 vars in 1093 bytes} [Sat Sep 10 23:42:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:47.462006 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc26eb-7a97-42ff-9c8c-63b787f006c2 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:47.572206 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:47.587036 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404123a7-120a-41a1-ab3d-c779f2b1a376 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:47.610018 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc26eb-7a97-42ff-9c8c-63b787f006c2 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:47.633702 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a52d2-e145-48e5-9756-fd0f3d50683d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:47.634492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a52d2-e145-48e5-9756-fd0f3d50683d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:47.634797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a52d2-e145-48e5-9756-fd0f3d50683d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:47.635390 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a52d2-e145-48e5-9756-fd0f3d50683d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:47.647926 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a52d2-e145-48e5-9756-fd0f3d50683d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:47.648821 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c52f80-0e2e-44f5-bcde-3f9c23be4d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:47.649033 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a52d2-e145-48e5-9756-fd0f3d50683d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:47.650005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c52f80-0e2e-44f5-bcde-3f9c23be4d9d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:47.650492 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c52f80-0e2e-44f5-bcde-3f9c23be4d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:47.650968 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c52f80-0e2e-44f5-bcde-3f9c23be4d9d None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:47.651621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:47.651953 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c52f80-0e2e-44f5-bcde-3f9c23be4d9d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:47.652267 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c52f80-0e2e-44f5-bcde-3f9c23be4d9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:47.653938 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c52f80-0e2e-44f5-bcde-3f9c23be4d9d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:47.665267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-f96a52d2-e145-48e5-9756-fd0f3d50683d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:47.665267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:47.665267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:47.665267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:47.665267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:47.665267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:47.665267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:47.665267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:47.665267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:47.665267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:47.665267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:47.665267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:47.665267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:47.665267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:47.665267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:47.665267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:47.666344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:47.666344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:47.666344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:47.666344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:47.666344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:47.666344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:47.666344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:47.666344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:47.666344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:47.666344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:47.666344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:47.666344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:47.666344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:47.666344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:47.666344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:47.666344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:47.666344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:47.667276 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:47.667276 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:47.667276 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:47.667276 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 383/742] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:47.668475 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631bdff-e7f0-492f-a2a3-ff87f5afa73e None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:47.668574 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631bdff-e7f0-492f-a2a3-ff87f5afa73e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:47.669268 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631bdff-e7f0-492f-a2a3-ff87f5afa73e None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:47.672636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 384/743] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:47.672876 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 360/744] 10.222.0.17 () {56 vars in 1133 bytes} [Sat Sep 10 23:42:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:47.675021 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2631bdff-e7f0-492f-a2a3-ff87f5afa73e None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:47.679083 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725f777-2dd9-4b9e-ad62-1a1fe1afacd3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:47.679366 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725f777-2dd9-4b9e-ad62-1a1fe1afacd3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:47.679632 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725f777-2dd9-4b9e-ad62-1a1fe1afacd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:47.699865 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-3725f777-2dd9-4b9e-ad62-1a1fe1afacd3 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:47.701050 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3725f777-2dd9-4b9e-ad62-1a1fe1afacd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:47.786693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 361/745] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:42:47.786693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725f777-2dd9-4b9e-ad62-1a1fe1afacd3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:47.820387 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2631bdff-e7f0-492f-a2a3-ff87f5afa73e None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:47.841812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:47.893741 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a10b27-f039-4275-8ed1-d2d84c603fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:47.894456 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a10b27-f039-4275-8ed1-d2d84c603fdd None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:47.894682 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a10b27-f039-4275-8ed1-d2d84c603fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:47.895003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a10b27-f039-4275-8ed1-d2d84c603fdd None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:47.895693 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a10b27-f039-4275-8ed1-d2d84c603fdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:47.895994 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a10b27-f039-4275-8ed1-d2d84c603fdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:47.897347 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a10b27-f039-4275-8ed1-d2d84c603fdd None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:47.907640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725f777-2dd9-4b9e-ad62-1a1fe1afacd3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:47.924184 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 385/746] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:47.929709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a10b27-f039-4275-8ed1-d2d84c603fdd None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:47.991971 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:47.992295 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9167d-05dc-472b-ba8e-9d9c9082c13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:47.992945 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9167d-05dc-472b-ba8e-9d9c9082c13b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:47.993269 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9167d-05dc-472b-ba8e-9d9c9082c13b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:47.993487 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9167d-05dc-472b-ba8e-9d9c9082c13b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:48.015755 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc9167d-05dc-472b-ba8e-9d9c9082c13b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:48.017495 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc9167d-05dc-472b-ba8e-9d9c9082c13b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:48.039662 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-8bc9167d-05dc-472b-ba8e-9d9c9082c13b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:48.039662 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:48.039662 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:48.039662 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:48.039662 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:48.039662 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:48.039662 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:48.039662 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:48.039662 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:48.039662 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:48.039662 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:48.039662 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:48.039662 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:48.039662 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:48.039662 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:48.039662 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:48.040841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:48.040841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:48.040841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:48.040841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:48.040841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:48.040841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:48.040841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:48.040841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:48.040841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:48.040841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:48.040841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:48.040841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:48.040841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:48.040841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:48.040841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:48.040841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:48.040841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:48.041967 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:48.041967 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:48.041967 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:48.045875 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 362/747] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:48.051884 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9167d-05dc-472b-ba8e-9d9c9082c13b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:48.073176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a10b27-f039-4275-8ed1-d2d84c603fdd None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:48.135912 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93834430-1ca0-4d9a-995a-915f39237dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:48.136652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93834430-1ca0-4d9a-995a-915f39237dee None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:48.137045 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93834430-1ca0-4d9a-995a-915f39237dee None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:48.137415 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93834430-1ca0-4d9a-995a-915f39237dee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:48.144463 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93834430-1ca0-4d9a-995a-915f39237dee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:48.146027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93834430-1ca0-4d9a-995a-915f39237dee None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:48.159003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-93834430-1ca0-4d9a-995a-915f39237dee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:48.159003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:48.159003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:48.159003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:48.159003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:48.159003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:48.159003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:48.159003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:48.159003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:48.159003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:48.159003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:48.159003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:48.159003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:42:48.159003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:42:48.159003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:42:48.160038 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:42:48.160038 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:48.160038 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:48.160038 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:48.160038 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:48.160038 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:48.160038 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:48.160038 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:48.160038 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:48.160038 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:48.160038 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:48.160038 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:48.160038 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:48.160038 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:48.160038 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:48.160038 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:48.160038 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:48.160920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:42:48.160920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:42:48.160920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:42:48.160920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:42:48.160920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:42:48.160920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:42:48.160920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:48.160920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:48.161336 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 386/748] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:48.162384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93834430-1ca0-4d9a-995a-915f39237dee None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:48.205771 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9167d-05dc-472b-ba8e-9d9c9082c13b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:48.221126 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:48.246481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30edf89-8caf-44cd-967e-8617723ed8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:48.247264 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30edf89-8caf-44cd-967e-8617723ed8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:48.247610 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30edf89-8caf-44cd-967e-8617723ed8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:48.248141 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30edf89-8caf-44cd-967e-8617723ed8b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:48.249271 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30edf89-8caf-44cd-967e-8617723ed8b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:48.250041 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30edf89-8caf-44cd-967e-8617723ed8b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:48.251959 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30edf89-8caf-44cd-967e-8617723ed8b5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:48.265186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 363/749] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:48.266233 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30edf89-8caf-44cd-967e-8617723ed8b5 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:48.283551 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93834430-1ca0-4d9a-995a-915f39237dee None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:48.307729 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:48.334493 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd3cf0-1ef3-4caf-937a-5f5f30a50172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:48.335386 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd3cf0-1ef3-4caf-937a-5f5f30a50172 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:48.335914 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd3cf0-1ef3-4caf-937a-5f5f30a50172 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:48.336513 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd3cf0-1ef3-4caf-937a-5f5f30a50172 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:48.351818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd3cf0-1ef3-4caf-937a-5f5f30a50172 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:48.359795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-c3fd3cf0-1ef3-4caf-937a-5f5f30a50172 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:48.359795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:48.359795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:48.359795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:48.359795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:48.359795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:48.359795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:48.359795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:48.359795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:48.359795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:48.359795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:48.359795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:48.359795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:48.359795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:48.359795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:48.359795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:48.360898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:48.360898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:48.360898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:48.360898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:48.360898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:48.360898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:48.360898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:48.360898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:48.360898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:48.360898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:48.360898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:48.360898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:48.360898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:48.360898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:48.360898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:48.360898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:48.361937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:48.361937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:48.361937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:48.361937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:48.361937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:48.361937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:48.361937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:48.361937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:48.361937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:48.361937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:48.361937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:48.361937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:48.364675 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 387/750] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Sep 10 23:42:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:48.365636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd3cf0-1ef3-4caf-937a-5f5f30a50172 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:48.395889 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30edf89-8caf-44cd-967e-8617723ed8b5 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:48.444995 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2caff5-76c9-49cd-ab29-e4f01ea4b3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:48.446158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2caff5-76c9-49cd-ab29-e4f01ea4b3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:48.446584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2caff5-76c9-49cd-ab29-e4f01ea4b3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:48.447010 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2caff5-76c9-49cd-ab29-e4f01ea4b3f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:48.448015 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2caff5-76c9-49cd-ab29-e4f01ea4b3f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:48.448487 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2caff5-76c9-49cd-ab29-e4f01ea4b3f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:48.451578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2caff5-76c9-49cd-ab29-e4f01ea4b3f6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:48.466842 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 364/751] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:42:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:48.468191 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2caff5-76c9-49cd-ab29-e4f01ea4b3f6 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:48.492292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd3cf0-1ef3-4caf-937a-5f5f30a50172 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:48.511790 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:48.534479 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c729d7-46a7-43fd-acc7-3f1c1b797aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:48.535332 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c729d7-46a7-43fd-acc7-3f1c1b797aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:48.535770 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c729d7-46a7-43fd-acc7-3f1c1b797aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:48.536310 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c729d7-46a7-43fd-acc7-3f1c1b797aa9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:48.546307 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c729d7-46a7-43fd-acc7-3f1c1b797aa9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:48.548231 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c729d7-46a7-43fd-acc7-3f1c1b797aa9 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:48.562277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-a5c729d7-46a7-43fd-acc7-3f1c1b797aa9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:48.562277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:48.562277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:48.562277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:48.562277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:48.562277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:48.562277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:48.562277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:48.562277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:48.562277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:48.562277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:48.562277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:48.562277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:42:48.562277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:42:48.562277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:42:48.563244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:42:48.563244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:48.563244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:48.563244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:48.563244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:48.563244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:48.563244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:48.563244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:48.563244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:48.563244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:48.563244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:48.563244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:48.563244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:48.563244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:48.563244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:48.563244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:48.563244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:48.564110 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:42:48.564110 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:42:48.564110 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:42:48.564110 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:42:48.564110 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:42:48.564110 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:42:48.564110 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:48.564110 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:48.566495 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 388/752] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:48.567573 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c729d7-46a7-43fd-acc7-3f1c1b797aa9 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:48.604748 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2caff5-76c9-49cd-ab29-e4f01ea4b3f6 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:48.608481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:48.653291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fdec91-0ca9-4d5a-a535-70223b3ae4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:48.653824 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdec91-0ca9-4d5a-a535-70223b3ae4fa None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:48.654108 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdec91-0ca9-4d5a-a535-70223b3ae4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:48.654382 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdec91-0ca9-4d5a-a535-70223b3ae4fa None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:48.655089 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fdec91-0ca9-4d5a-a535-70223b3ae4fa None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:48.655391 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fdec91-0ca9-4d5a-a535-70223b3ae4fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:48.656777 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fdec91-0ca9-4d5a-a535-70223b3ae4fa None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:48.670755 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 365/753] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Sep 10 23:42:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:48.671723 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fdec91-0ca9-4d5a-a535-70223b3ae4fa None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:48.691377 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c729d7-46a7-43fd-acc7-3f1c1b797aa9 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:48.730507 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063d496-f140-42ab-aac9-0c7b45c5bc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:48.731465 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063d496-f140-42ab-aac9-0c7b45c5bc0e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:48.731744 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063d496-f140-42ab-aac9-0c7b45c5bc0e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:48.732012 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063d496-f140-42ab-aac9-0c7b45c5bc0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:48.732823 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4063d496-f140-42ab-aac9-0c7b45c5bc0e None admin] RBAC: Adding query filter params (role.id=9dce48a2b53c482d931dcd064b07744e, scope.project.id=947277c4458645899287e6818947c076, user.id=564e65585e9543e286b6041af870cf41) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:48.733132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4063d496-f140-42ab-aac9-0c7b45c5bc0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:48.734774 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4063d496-f140-42ab-aac9-0c7b45c5bc0e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:48.749274 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 389/754] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:42:48 2022] GET /identity/v3/role_assignments?user.id=564e65585e9543e286b6041af870cf41&scope.project.id=947277c4458645899287e6818947c076&role.id=9dce48a2b53c482d931dcd064b07744e => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:48.751805 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063d496-f140-42ab-aac9-0c7b45c5bc0e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:48.782907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:48.792211 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fdec91-0ca9-4d5a-a535-70223b3ae4fa None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:48.829589 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbf660-e0ee-4467-8dd4-5ca500f8d4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:48.830199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbf660-e0ee-4467-8dd4-5ca500f8d4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:48.830473 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbf660-e0ee-4467-8dd4-5ca500f8d4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:48.830762 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbf660-e0ee-4467-8dd4-5ca500f8d4b5 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:48.831447 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fbf660-e0ee-4467-8dd4-5ca500f8d4b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:48.831742 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fbf660-e0ee-4467-8dd4-5ca500f8d4b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:48.833163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fbf660-e0ee-4467-8dd4-5ca500f8d4b5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:48.846424 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 366/755] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:42:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:48.856517 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbf660-e0ee-4467-8dd4-5ca500f8d4b5 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:48.872040 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:48.875936 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063d496-f140-42ab-aac9-0c7b45c5bc0e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:48.922682 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136bee9-c953-46af-a316-1eb274934447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:48.927788 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136bee9-c953-46af-a316-1eb274934447 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:48.928070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136bee9-c953-46af-a316-1eb274934447 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:48.928333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136bee9-c953-46af-a316-1eb274934447 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:48.953478 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d136bee9-c953-46af-a316-1eb274934447 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:48.955019 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d136bee9-c953-46af-a316-1eb274934447 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:48.975748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-d136bee9-c953-46af-a316-1eb274934447 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:48.975748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:48.975748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:48.975748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:48.975748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:48.975748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:48.975748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:48.975748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:48.975748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:48.975748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:48.975748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:48.975748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:48.975748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:48.975748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:48.975748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:48.975748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:48.977873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:48.977873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:48.977873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:48.977873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:48.977873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:48.977873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:48.977873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:48.977873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:48.977873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:48.977873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:48.977873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:48.977873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:48.977873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:48.977873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:48.977873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:48.977873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:48.977873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:48.979289 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:48.979289 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:48.979289 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:48.982481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 390/756] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:48.983847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63406f-03b0-4a7e-8525-4df0f0760df7 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:48.984126 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63406f-03b0-4a7e-8525-4df0f0760df7 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:48.984394 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63406f-03b0-4a7e-8525-4df0f0760df7 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:48.986251 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 391/757] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:48.987011 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63406f-03b0-4a7e-8525-4df0f0760df7 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:49.033238 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbf660-e0ee-4467-8dd4-5ca500f8d4b5 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:49.072089 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e0d73-1bee-48d5-bde2-0b9b08b99e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:49.073020 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e0d73-1bee-48d5-bde2-0b9b08b99e28 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:49.073368 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e0d73-1bee-48d5-bde2-0b9b08b99e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:49.074293 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e0d73-1bee-48d5-bde2-0b9b08b99e28 None admin] PATH_INFO: `/v3/projects/947277c4458645899287e6818947c076/users/636bfafb693044d283530063047fc0c2/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:49.076295 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e0d73-1bee-48d5-bde2-0b9b08b99e28 None admin] RBAC: Authorizing `identity:create_grant(project_id=947277c4458645899287e6818947c076, user_id=636bfafb693044d283530063047fc0c2, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:49.078719 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e0d73-1bee-48d5-bde2-0b9b08b99e28 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:49.116436 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63406f-03b0-4a7e-8525-4df0f0760df7 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:49.151469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f396d15-c2cb-4ac8-b33f-c4b452b6fa8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:49.152256 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f396d15-c2cb-4ac8-b33f-c4b452b6fa8d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:49.152526 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f396d15-c2cb-4ac8-b33f-c4b452b6fa8d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:49.152791 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f396d15-c2cb-4ac8-b33f-c4b452b6fa8d None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:49.153444 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f396d15-c2cb-4ac8-b33f-c4b452b6fa8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:49.153727 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f396d15-c2cb-4ac8-b33f-c4b452b6fa8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:49.155485 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f396d15-c2cb-4ac8-b33f-c4b452b6fa8d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:49.164268 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 367/758] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Sep 10 23:42:48 2022] PUT /identity/v3/projects/947277c4458645899287e6818947c076/users/636bfafb693044d283530063047fc0c2/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:42:49.167040 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6802856-0908-4744-859f-9516507ed6b3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:49.168342 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6802856-0908-4744-859f-9516507ed6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:49.168958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6802856-0908-4744-859f-9516507ed6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:49.173880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 392/759] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:49.175106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f396d15-c2cb-4ac8-b33f-c4b452b6fa8d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:49.192726 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:49.197102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a6802856-0908-4744-859f-9516507ed6b3 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:49.199503 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a6802856-0908-4744-859f-9516507ed6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:49.210300 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:49.331716 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 368/760] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:42:49.340053 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f396d15-c2cb-4ac8-b33f-c4b452b6fa8d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:49.409629 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d1102-54a1-4bad-926e-0daa7c7e5adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:49.410611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d1102-54a1-4bad-926e-0daa7c7e5adb None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:49.411098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d1102-54a1-4bad-926e-0daa7c7e5adb None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:49.411542 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d1102-54a1-4bad-926e-0daa7c7e5adb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:49.422722 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d1102-54a1-4bad-926e-0daa7c7e5adb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:49.424498 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d1102-54a1-4bad-926e-0daa7c7e5adb None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:49.445446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-d07d1102-54a1-4bad-926e-0daa7c7e5adb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 23:42:49.445446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:49.445446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:49.445446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:49.445446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:49.445446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:49.445446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:49.445446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:49.445446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:49.445446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:49.445446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:49.445446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:49.445446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:42:49.445446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:42:49.445446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:42:49.445446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:42:49.446417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:49.446417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:49.446417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:42:49.446417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:49.446417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:42:49.446417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:49.446417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:49.446417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:49.446417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:49.446417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:49.446417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:49.446417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:49.446417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:49.446417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:49.446417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:49.446417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:49.446417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:49.446417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:49.447413 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:49.447413 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:49.447413 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:42:49.447413 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:42:49.447413 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:42:49.447413 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:42:49.447413 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:42:49.447413 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:42:49.447413 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 23:42:49.447413 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:49.449628 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 393/761] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Sep 10 23:42:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:49.455059 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6802856-0908-4744-859f-9516507ed6b3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:49.564951 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6802856-0908-4744-859f-9516507ed6b3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:49.602191 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c822b6c-d49d-48cb-b2b4-e2e4b7cdd957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:49.602828 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c822b6c-d49d-48cb-b2b4-e2e4b7cdd957 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:49.603108 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c822b6c-d49d-48cb-b2b4-e2e4b7cdd957 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:49.603420 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c822b6c-d49d-48cb-b2b4-e2e4b7cdd957 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:49.604149 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c822b6c-d49d-48cb-b2b4-e2e4b7cdd957 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:49.604447 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c822b6c-d49d-48cb-b2b4-e2e4b7cdd957 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:49.606842 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c822b6c-d49d-48cb-b2b4-e2e4b7cdd957 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:49.632021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:49.635011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 369/762] 10.222.0.17 () {56 vars in 1149 bytes} [Sat Sep 10 23:42:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:49.637951 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d1102-54a1-4bad-926e-0daa7c7e5adb None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:49.670698 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c822b6c-d49d-48cb-b2b4-e2e4b7cdd957 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:49.804834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c822b6c-d49d-48cb-b2b4-e2e4b7cdd957 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:49.879384 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd8502-4772-46ba-a293-931427147cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:49.880017 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd8502-4772-46ba-a293-931427147cba None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:49.880290 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd8502-4772-46ba-a293-931427147cba None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:49.880558 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd8502-4772-46ba-a293-931427147cba None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:49.881146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd8502-4772-46ba-a293-931427147cba None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:49.882506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd8502-4772-46ba-a293-931427147cba None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:49.884426 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:49.904165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-c6dd8502-4772-46ba-a293-931427147cba None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 23:42:49.904165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:49.904165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 23:42:49.904165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 23:42:49.904165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 23:42:49.904165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 23:42:49.904165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 23:42:49.904165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:42:49.904165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 23:42:49.904165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:49.904165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:42:49.904165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:49.904165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:49.904165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:49.904165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:49.904165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:49.904165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:49.905355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:49.905355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:49.905355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:49.905355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:49.905355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:49.905355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:49.905355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 23:42:49.905355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 23:42:49.905355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 23:42:49.905355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 23:42:49.905355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:49.905355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:49.905355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:49.905355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:49.905355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:49.905355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:49.906199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:49.906199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:49.906199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:49.906199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:49.906199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:49.906199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:49.906199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:49.906199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:49.906199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:49.906199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:49.906199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 23:42:49.906199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:42:49.906199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 23:42:49.906199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:49.906199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 370/763] 10.222.0.17 () {56 vars in 1129 bytes} [Sat Sep 10 23:42:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:49.911092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd8502-4772-46ba-a293-931427147cba None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:49.916566 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d1102-54a1-4bad-926e-0daa7c7e5adb None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:49.955310 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cdc065-ce7e-44e1-915e-246de6f113ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:49.956079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cdc065-ce7e-44e1-915e-246de6f113ab None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:49.956240 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cdc065-ce7e-44e1-915e-246de6f113ab None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:49.956432 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cdc065-ce7e-44e1-915e-246de6f113ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:50.016111 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:50.041742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cdc065-ce7e-44e1-915e-246de6f113ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:50.043803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cdc065-ce7e-44e1-915e-246de6f113ab None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:50.058319 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd8502-4772-46ba-a293-931427147cba None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:50.095557 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfb890-8a2b-476d-b35c-12c6ee4045ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:50.096266 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfb890-8a2b-476d-b35c-12c6ee4045ef None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:50.096596 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfb890-8a2b-476d-b35c-12c6ee4045ef None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:50.096872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfb890-8a2b-476d-b35c-12c6ee4045ef None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:50.097503 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfb890-8a2b-476d-b35c-12c6ee4045ef None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:50.097804 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfb890-8a2b-476d-b35c-12c6ee4045ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:50.099336 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfb890-8a2b-476d-b35c-12c6ee4045ef None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:50.104423 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 371/764] 10.222.0.17 () {56 vars in 1124 bytes} [Sat Sep 10 23:42:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:42:50.110927 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96defc28-3578-48a4-84bc-b6653c7d153d None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:50.111213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96defc28-3578-48a4-84bc-b6653c7d153d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:50.111497 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96defc28-3578-48a4-84bc-b6653c7d153d None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:50.112902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 372/765] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:50.113700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96defc28-3578-48a4-84bc-b6653c7d153d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:50.121646 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:50.162748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 394/766] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:42:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 23:42:50.165899 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04a3dc-73c6-4a1f-a60c-8114ec39121e None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:50.166282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04a3dc-73c6-4a1f-a60c-8114ec39121e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:50.166504 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04a3dc-73c6-4a1f-a60c-8114ec39121e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:50.190759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-bc04a3dc-73c6-4a1f-a60c-8114ec39121e None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:50.192285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-bc04a3dc-73c6-4a1f-a60c-8114ec39121e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:50.237690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96defc28-3578-48a4-84bc-b6653c7d153d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:50.266577 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:50.275329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76809394-177a-47f7-bad6-c3c4408d9cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:50.275993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76809394-177a-47f7-bad6-c3c4408d9cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:50.281234 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76809394-177a-47f7-bad6-c3c4408d9cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:50.281234 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76809394-177a-47f7-bad6-c3c4408d9cc4 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:50.281407 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 395/767] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:42:50.281482 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76809394-177a-47f7-bad6-c3c4408d9cc4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:50.281482 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76809394-177a-47f7-bad6-c3c4408d9cc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:50.281482 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76809394-177a-47f7-bad6-c3c4408d9cc4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:50.283584 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04a3dc-73c6-4a1f-a60c-8114ec39121e None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:50.289617 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 373/768] 10.222.0.17 () {56 vars in 1124 bytes} [Sat Sep 10 23:42:50 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:50.302717 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76809394-177a-47f7-bad6-c3c4408d9cc4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:50.456504 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76809394-177a-47f7-bad6-c3c4408d9cc4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:50.493922 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04a3dc-73c6-4a1f-a60c-8114ec39121e None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:50.498959 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:50.501973 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c55a334-3065-46f3-af8a-5408cf0bc694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:50.502628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55a334-3065-46f3-af8a-5408cf0bc694 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:50.502999 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55a334-3065-46f3-af8a-5408cf0bc694 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:50.503282 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55a334-3065-46f3-af8a-5408cf0bc694 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:50.504094 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c55a334-3065-46f3-af8a-5408cf0bc694 None admin] RBAC: Adding query filter params (interface=public, service_id=426e789c373f45f0b53011d772b942b4, region_id=RegionOne) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:50.504411 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c55a334-3065-46f3-af8a-5408cf0bc694 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:50.505871 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c55a334-3065-46f3-af8a-5408cf0bc694 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:50.515246 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 374/769] 10.222.0.17 () {56 vars in 1251 bytes} [Sat Sep 10 23:42:50 2022] GET /identity/v3/endpoints?service_id=426e789c373f45f0b53011d772b942b4&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:50.520576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c55a334-3065-46f3-af8a-5408cf0bc694 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:50.588892 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:50.608405 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec0356-7c4b-4910-b346-7d86df30b227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:50.609040 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec0356-7c4b-4910-b346-7d86df30b227 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:50.609316 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec0356-7c4b-4910-b346-7d86df30b227 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:50.609580 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec0356-7c4b-4910-b346-7d86df30b227 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:50.626464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec0356-7c4b-4910-b346-7d86df30b227 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:50.627944 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec0356-7c4b-4910-b346-7d86df30b227 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:50.646918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-ddec0356-7c4b-4910-b346-7d86df30b227 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:50.646918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:50.646918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:50.646918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:50.646918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:50.646918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:50.646918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:50.646918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:50.646918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:50.646918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:50.646918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:50.646918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:50.646918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:50.646918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:50.646918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:50.646918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:50.647909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:50.647909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:50.647909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:50.647909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:50.647909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:50.647909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:50.647909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:50.647909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:50.647909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:50.647909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:50.647909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:50.647909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:50.647909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:50.647909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:50.647909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:50.647909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:50.647909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:50.648779 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:50.648779 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:50.648779 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:50.648779 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 396/770] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:50.649152 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec0356-7c4b-4910-b346-7d86df30b227 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:50.656169 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c55a334-3065-46f3-af8a-5408cf0bc694 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:50.692332 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13de22c-1a0f-4fda-94f3-81f91b6fdf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:50.693202 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13de22c-1a0f-4fda-94f3-81f91b6fdf20 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:50.693621 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13de22c-1a0f-4fda-94f3-81f91b6fdf20 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:50.694030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13de22c-1a0f-4fda-94f3-81f91b6fdf20 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:50.694798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13de22c-1a0f-4fda-94f3-81f91b6fdf20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:50.696305 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13de22c-1a0f-4fda-94f3-81f91b6fdf20 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:50.710774 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 375/771] 10.222.0.17 () {56 vars in 1087 bytes} [Sat Sep 10 23:42:50 2022] GET /identity/v3/services => generated 772 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:50.712009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13de22c-1a0f-4fda-94f3-81f91b6fdf20 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:50.779249 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec0356-7c4b-4910-b346-7d86df30b227 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:50.865968 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc691a7-dec7-4651-b25a-8be859c7373b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:50.866532 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc691a7-dec7-4651-b25a-8be859c7373b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:50.866840 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc691a7-dec7-4651-b25a-8be859c7373b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:50.867109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc691a7-dec7-4651-b25a-8be859c7373b None admin] PATH_INFO: `/v3/services/network` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:50.867795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc691a7-dec7-4651-b25a-8be859c7373b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:50.869165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc691a7-dec7-4651-b25a-8be859c7373b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:50.885981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-7bc691a7-dec7-4651-b25a-8be859c7373b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 23:42:50.885981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:50.885981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 23:42:50.885981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 23:42:50.885981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 23:42:50.885981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 23:42:50.885981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 23:42:50.885981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:42:50.885981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 23:42:50.885981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:50.885981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:42:50.885981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:50.885981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:50.885981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:50.885981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:50.885981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:50.885981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:50.887604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:50.887604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:50.887604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:50.887604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:50.887604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:50.887604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:50.887604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 23:42:50.887604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 23:42:50.887604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 23:42:50.887604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 23:42:50.887604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:50.887604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:50.887604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:50.887604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:50.887604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:50.887604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:50.888441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:50.888441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:50.888441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:50.888441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:50.888441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:50.888441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:50.888441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:50.888441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:50.888441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:50.888441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:50.888441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 23:42:50.888441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:42:50.888441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 23:42:50.888441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:50.888441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 397/772] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:50 2022] GET /identity/v3/services/network => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:50.892059 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc691a7-dec7-4651-b25a-8be859c7373b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:50.909680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13de22c-1a0f-4fda-94f3-81f91b6fdf20 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:50.936627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:50.939365 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:50.956325 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eac935-caba-4ff1-82d6-84ec47cd82c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:50.956930 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eac935-caba-4ff1-82d6-84ec47cd82c5 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:50.957210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eac935-caba-4ff1-82d6-84ec47cd82c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:50.957469 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eac935-caba-4ff1-82d6-84ec47cd82c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:50.958106 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eac935-caba-4ff1-82d6-84ec47cd82c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:50.958403 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eac935-caba-4ff1-82d6-84ec47cd82c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:50.960964 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eac935-caba-4ff1-82d6-84ec47cd82c5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:50.978321 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 376/773] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:50.979456 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eac935-caba-4ff1-82d6-84ec47cd82c5 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:51.045575 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc691a7-dec7-4651-b25a-8be859c7373b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:51.089750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a0778-f3e8-4b1d-8d55-ef135cf8dce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:51.090768 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a0778-f3e8-4b1d-8d55-ef135cf8dce1 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:51.091208 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a0778-f3e8-4b1d-8d55-ef135cf8dce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:51.091777 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a0778-f3e8-4b1d-8d55-ef135cf8dce1 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:51.092624 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a0778-f3e8-4b1d-8d55-ef135cf8dce1 None admin] RBAC: Adding query filter params (name=network) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:51.093113 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a0778-f3e8-4b1d-8d55-ef135cf8dce1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:51.094858 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a0778-f3e8-4b1d-8d55-ef135cf8dce1 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:51.102121 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 398/774] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Sep 10 23:42:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:51.113534 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a0778-f3e8-4b1d-8d55-ef135cf8dce1 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:51.115333 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eac935-caba-4ff1-82d6-84ec47cd82c5 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:51.154640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55a9d7-7e05-4e6a-b9f6-be8412519d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:51.155361 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55a9d7-7e05-4e6a-b9f6-be8412519d7b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:51.155683 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55a9d7-7e05-4e6a-b9f6-be8412519d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:51.155949 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55a9d7-7e05-4e6a-b9f6-be8412519d7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:51.163861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e55a9d7-7e05-4e6a-b9f6-be8412519d7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:51.165551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e55a9d7-7e05-4e6a-b9f6-be8412519d7b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:51.178392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-1e55a9d7-7e05-4e6a-b9f6-be8412519d7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:51.178392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:51.178392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:51.178392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:51.178392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:51.178392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:51.178392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:51.178392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:51.178392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:51.178392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:51.178392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:51.178392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:51.178392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:42:51.178392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:42:51.178392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:42:51.179422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:42:51.179422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:51.179422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:51.179422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:51.179422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:51.179422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:51.179422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:51.179422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:51.179422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:51.179422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:51.179422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:51.179422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:51.179422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:51.179422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:51.179422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:51.179422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:51.179422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:51.180303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:42:51.180303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:42:51.180303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:42:51.180303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:42:51.180303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:42:51.180303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:42:51.180303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:51.180303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:51.180303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 377/775] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:42:51.190154 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55a9d7-7e05-4e6a-b9f6-be8412519d7b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:51.256889 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:51.269638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:51.291543 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a0778-f3e8-4b1d-8d55-ef135cf8dce1 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:51.337959 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d79d7-ff75-4bb7-892a-70661ccbc349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:51.338606 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d79d7-ff75-4bb7-892a-70661ccbc349 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:51.338871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d79d7-ff75-4bb7-892a-70661ccbc349 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:51.339667 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d79d7-ff75-4bb7-892a-70661ccbc349 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:51.340357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d79d7-ff75-4bb7-892a-70661ccbc349 None admin] RBAC: Adding query filter params (type=network) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:51.340662 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d79d7-ff75-4bb7-892a-70661ccbc349 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:51.342161 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d79d7-ff75-4bb7-892a-70661ccbc349 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:51.345172 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55a9d7-7e05-4e6a-b9f6-be8412519d7b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:51.347670 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 399/776] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Sep 10 23:42:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:51.349070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f713d-77dd-4736-ad9c-7664d50926c5 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:51.349516 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f713d-77dd-4736-ad9c-7664d50926c5 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:51.349933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f713d-77dd-4736-ad9c-7664d50926c5 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:51.354743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 400/777] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:51.354743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f713d-77dd-4736-ad9c-7664d50926c5 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:51.396424 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8187002-7da3-4645-a184-f1e9e3ee40a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:51.397280 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8187002-7da3-4645-a184-f1e9e3ee40a7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:51.397697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8187002-7da3-4645-a184-f1e9e3ee40a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:51.398697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8187002-7da3-4645-a184-f1e9e3ee40a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:51.399541 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8187002-7da3-4645-a184-f1e9e3ee40a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:51.399993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8187002-7da3-4645-a184-f1e9e3ee40a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:51.402785 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8187002-7da3-4645-a184-f1e9e3ee40a7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:51.420937 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 378/778] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:42:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:51.422386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d53a2-9e24-497a-aa35-64c0eb028aae None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:51.422860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d53a2-9e24-497a-aa35-64c0eb028aae None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:51.423268 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d53a2-9e24-497a-aa35-64c0eb028aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:51.445584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-975d53a2-9e24-497a-aa35-64c0eb028aae None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:51.446951 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-975d53a2-9e24-497a-aa35-64c0eb028aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:51.481757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f713d-77dd-4736-ad9c-7664d50926c5 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:51.522535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305dfcea-ba65-462b-8e27-c05db3b17cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:51.522905 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 379/779] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:42:51.523806 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dfcea-ba65-462b-8e27-c05db3b17cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:51.523937 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d53a2-9e24-497a-aa35-64c0eb028aae None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:51.524363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dfcea-ba65-462b-8e27-c05db3b17cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:51.524868 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dfcea-ba65-462b-8e27-c05db3b17cd1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:51.525721 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305dfcea-ba65-462b-8e27-c05db3b17cd1 None admin] RBAC: Adding query filter params (interface=public, service_id=6b7f7f6778fe4910886117fab26759df, region_id=RegionOne) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:51.526285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305dfcea-ba65-462b-8e27-c05db3b17cd1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:51.527933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305dfcea-ba65-462b-8e27-c05db3b17cd1 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:51.539804 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 401/780] 10.222.0.17 () {56 vars in 1251 bytes} [Sat Sep 10 23:42:51 2022] GET /identity/v3/endpoints?service_id=6b7f7f6778fe4910886117fab26759df&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:51.547691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305dfcea-ba65-462b-8e27-c05db3b17cd1 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:51.569148 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:51.584649 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:51.707332 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d53a2-9e24-497a-aa35-64c0eb028aae None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:51.719895 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305dfcea-ba65-462b-8e27-c05db3b17cd1 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:51.776039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb7df1a-fe8e-42bf-9025-e49398ea3160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:51.777183 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb7df1a-fe8e-42bf-9025-e49398ea3160 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:51.777627 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb7df1a-fe8e-42bf-9025-e49398ea3160 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:51.777988 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb7df1a-fe8e-42bf-9025-e49398ea3160 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:51.779200 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb7df1a-fe8e-42bf-9025-e49398ea3160 None admin] RBAC: Adding query filter params (role.id=8ff1727b65104e5fb97333d477196d46, scope.project.id=947277c4458645899287e6818947c076, user.id=b0a879085a194b0687e1cdcbafd48033) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:51.779508 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb7df1a-fe8e-42bf-9025-e49398ea3160 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:51.780939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb7df1a-fe8e-42bf-9025-e49398ea3160 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:51.784159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7760e-7113-45f0-b234-9b35fb642ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:51.784758 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7760e-7113-45f0-b234-9b35fb642ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:51.785023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7760e-7113-45f0-b234-9b35fb642ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:51.785291 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7760e-7113-45f0-b234-9b35fb642ee5 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:51.785883 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec7760e-7113-45f0-b234-9b35fb642ee5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:51.787232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec7760e-7113-45f0-b234-9b35fb642ee5 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:51.799794 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 380/781] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:42:51 2022] GET /identity/v3/role_assignments?user.id=b0a879085a194b0687e1cdcbafd48033&scope.project.id=947277c4458645899287e6818947c076&role.id=8ff1727b65104e5fb97333d477196d46 => generated 259 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:51.800094 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 402/782] 10.222.0.17 () {56 vars in 1087 bytes} [Sat Sep 10 23:42:51 2022] GET /identity/v3/services => generated 772 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:42:51.846733 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b74af-bf2b-49a9-9576-4be40bc8359e None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:51.847079 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b74af-bf2b-49a9-9576-4be40bc8359e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:51.847289 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b74af-bf2b-49a9-9576-4be40bc8359e None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:51.849155 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 381/783] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:51.855514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7713c-4798-4dc2-ad2a-3202f09c544c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:51.855859 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7713c-4798-4dc2-ad2a-3202f09c544c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:51.856072 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7713c-4798-4dc2-ad2a-3202f09c544c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:51.883323 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-43a7713c-4798-4dc2-ad2a-3202f09c544c None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:51.884569 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-43a7713c-4798-4dc2-ad2a-3202f09c544c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:51.891836 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b74af-bf2b-49a9-9576-4be40bc8359e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:52.031127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 403/784] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:42:52.082091 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:52.090241 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b74af-bf2b-49a9-9576-4be40bc8359e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:52.124664 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360c70c-5c79-45e3-bd5f-6aab714ecd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:52.125490 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360c70c-5c79-45e3-bd5f-6aab714ecd9e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:52.125768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360c70c-5c79-45e3-bd5f-6aab714ecd9e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:52.126042 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360c70c-5c79-45e3-bd5f-6aab714ecd9e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:52.137937 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8360c70c-5c79-45e3-bd5f-6aab714ecd9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:52.144734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-8360c70c-5c79-45e3-bd5f-6aab714ecd9e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:42:52.144734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:52.144734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:52.144734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:52.144734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:52.144734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:52.144734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:52.144734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:52.144734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:52.144734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:52.144734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:52.144734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:52.144734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:52.144734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:52.144734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:52.144734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:52.145676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:52.145676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:52.145676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:52.145676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:52.145676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:52.145676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:52.145676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:52.145676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:52.145676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:52.145676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:52.145676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:52.145676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:52.145676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:52.145676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:52.145676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:52.145676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:52.146514 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:52.146514 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:52.146514 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:52.146514 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:52.146514 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:52.146514 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:52.146514 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:52.146514 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:52.146514 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:52.146514 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:52.146514 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:42:52.146514 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:52.146514 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 382/785] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:42:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:42:52.154254 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7713c-4798-4dc2-ad2a-3202f09c544c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:52.161271 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:52.281991 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7713c-4798-4dc2-ad2a-3202f09c544c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:52.318549 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360c70c-5c79-45e3-bd5f-6aab714ecd9e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:52.339299 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85313d0-9f2e-41a2-b397-f86fd38d0fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:52.340692 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85313d0-9f2e-41a2-b397-f86fd38d0fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:52.340974 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85313d0-9f2e-41a2-b397-f86fd38d0fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:52.341243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85313d0-9f2e-41a2-b397-f86fd38d0fe7 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:52.341917 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85313d0-9f2e-41a2-b397-f86fd38d0fe7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:52.342215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85313d0-9f2e-41a2-b397-f86fd38d0fe7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:52.343732 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85313d0-9f2e-41a2-b397-f86fd38d0fe7 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:52.358713 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 404/786] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:42:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:52.363964 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85313d0-9f2e-41a2-b397-f86fd38d0fe7 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:52.447871 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360c70c-5c79-45e3-bd5f-6aab714ecd9e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:52.456228 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:52.495225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85313d0-9f2e-41a2-b397-f86fd38d0fe7 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:52.531368 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2907e-fe78-4727-bcdb-3564bf7fc2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:52.532055 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2907e-fe78-4727-bcdb-3564bf7fc2ee None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:52.532271 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2907e-fe78-4727-bcdb-3564bf7fc2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:52.532540 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2907e-fe78-4727-bcdb-3564bf7fc2ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:52.534238 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d8078-e448-4770-bace-e62ebc1b343d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:52.535167 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d8078-e448-4770-bace-e62ebc1b343d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:52.535644 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d8078-e448-4770-bace-e62ebc1b343d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:52.536107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d8078-e448-4770-bace-e62ebc1b343d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:52.547895 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d8078-e448-4770-bace-e62ebc1b343d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:52.549613 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d8078-e448-4770-bace-e62ebc1b343d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:52.565925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-1a5d8078-e448-4770-bace-e62ebc1b343d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:52.565925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:52.565925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:52.565925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:52.565925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:52.565925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:52.565925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:52.565925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:52.565925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:52.565925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:52.565925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:52.565925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:52.565925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:52.565925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:52.565925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:52.565925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:52.566953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:52.566953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:52.566953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:52.566953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:52.566953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:52.566953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:52.566953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:52.566953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:52.566953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:52.566953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:52.566953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:52.566953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:52.566953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:52.566953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:52.566953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:52.566953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:52.566953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:52.567842 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:52.567842 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:52.567842 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:52.568027 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2907e-fe78-4727-bcdb-3564bf7fc2ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:52.570234 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 405/787] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:52.574164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-2cf2907e-fe78-4727-bcdb-3564bf7fc2ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:42:52.574164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:52.574164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:52.574164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:52.574164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:52.574164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:52.574164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:52.574164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:52.574164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:52.574164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:52.574164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:52.574164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:52.574164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:52.574164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:52.574164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:52.574164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:52.576422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:52.576422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:52.576422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:52.576422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:52.576422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:52.576422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:52.576422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:52.576422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:52.576422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:52.576422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:52.576422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:52.576422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:52.576422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:52.576422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:52.576422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:52.576422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:52.577485 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:52.577485 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:52.577485 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:52.577485 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:52.577485 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:52.577485 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:52.577485 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:52.577485 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:52.577485 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:52.577485 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:52.577485 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:42:52.577485 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:52.577485 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 383/788] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:42:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:42:52.579282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d8078-e448-4770-bace-e62ebc1b343d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:52.595680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2907e-fe78-4727-bcdb-3564bf7fc2ee None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:52.622094 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:52.687747 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d8078-e448-4770-bace-e62ebc1b343d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:52.705938 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:52.724771 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91144991-b540-4b46-9f9b-0a1a4c43ab92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:52.725670 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91144991-b540-4b46-9f9b-0a1a4c43ab92 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:52.726157 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91144991-b540-4b46-9f9b-0a1a4c43ab92 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:52.726690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91144991-b540-4b46-9f9b-0a1a4c43ab92 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:52.727522 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91144991-b540-4b46-9f9b-0a1a4c43ab92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:52.728003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91144991-b540-4b46-9f9b-0a1a4c43ab92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:52.729552 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91144991-b540-4b46-9f9b-0a1a4c43ab92 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:52.742471 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 406/789] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:52.752392 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91144991-b540-4b46-9f9b-0a1a4c43ab92 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:52.788660 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2907e-fe78-4727-bcdb-3564bf7fc2ee None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:52.861223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91144991-b540-4b46-9f9b-0a1a4c43ab92 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:52.867353 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5434a9ce-5b87-415b-9479-f9ac2689c40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:52.867966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434a9ce-5b87-415b-9479-f9ac2689c40a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:52.868247 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434a9ce-5b87-415b-9479-f9ac2689c40a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:52.868522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434a9ce-5b87-415b-9479-f9ac2689c40a None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:52.869195 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5434a9ce-5b87-415b-9479-f9ac2689c40a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:52.869496 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5434a9ce-5b87-415b-9479-f9ac2689c40a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:52.871117 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5434a9ce-5b87-415b-9479-f9ac2689c40a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:52.890349 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 384/790] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:42:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:52.895598 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5434a9ce-5b87-415b-9479-f9ac2689c40a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:52.903977 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d512fe-4748-4c76-8b9d-4251ffa5d096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:52.906497 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d512fe-4748-4c76-8b9d-4251ffa5d096 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:52.907022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d512fe-4748-4c76-8b9d-4251ffa5d096 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:52.908068 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d512fe-4748-4c76-8b9d-4251ffa5d096 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:52.924873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d512fe-4748-4c76-8b9d-4251ffa5d096 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:52.927545 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d512fe-4748-4c76-8b9d-4251ffa5d096 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:52.955055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-a4d512fe-4748-4c76-8b9d-4251ffa5d096 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 10 23:42:52.955055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:52.955055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:52.955055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:52.955055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:52.955055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:52.955055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:52.955055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:52.955055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:52.955055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:52.955055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:52.955055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:52.955055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:42:52.955055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:42:52.955055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:42:52.955055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:42:52.956060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:52.956060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:52.956060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:42:52.956060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:52.956060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:42:52.956060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:52.956060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:52.956060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:52.956060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:52.956060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:52.956060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:52.956060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:52.956060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:52.956060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:52.956060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:52.956060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:52.956060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:52.956060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:52.957000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:52.957000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:52.957000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:42:52.957000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:42:52.957000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:42:52.957000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:42:52.957000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:42:52.957000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:42:52.957000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 10 23:42:52.957000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:52.959078 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 407/791] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:42:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:52.965899 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d512fe-4748-4c76-8b9d-4251ffa5d096 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:53.047144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:53.054987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:53.090603 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5434a9ce-5b87-415b-9479-f9ac2689c40a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:53.137108 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d512fe-4748-4c76-8b9d-4251ffa5d096 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:53.159884 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c997b84a-6e71-4b0a-8a6b-2817574bfaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:53.161225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997b84a-6e71-4b0a-8a6b-2817574bfaeb None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:53.163104 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997b84a-6e71-4b0a-8a6b-2817574bfaeb None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:53.165423 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997b84a-6e71-4b0a-8a6b-2817574bfaeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:53.193971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c997b84a-6e71-4b0a-8a6b-2817574bfaeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:53.197095 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c997b84a-6e71-4b0a-8a6b-2817574bfaeb None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:53.222221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d247b1-9960-44ec-bd27-4752ecfe3f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:53.224859 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d247b1-9960-44ec-bd27-4752ecfe3f53 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:53.225361 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d247b1-9960-44ec-bd27-4752ecfe3f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:53.226631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d247b1-9960-44ec-bd27-4752ecfe3f53 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:53.227587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d247b1-9960-44ec-bd27-4752ecfe3f53 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:53.231760 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-c997b84a-6e71-4b0a-8a6b-2817574bfaeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:53.231760 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:53.231760 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:53.231760 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:53.231760 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:53.231760 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:53.231760 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:53.231760 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:53.231760 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:53.231760 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:53.231760 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:53.231760 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:53.231760 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:53.231760 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:53.231760 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:53.231760 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:53.232680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:53.232680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:53.232680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:53.232680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:53.232680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:53.232680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:53.232680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:53.232680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:53.232680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:53.232680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:53.232680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:53.232680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:53.232680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:53.232680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:53.232680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:53.232680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:53.232680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:53.233568 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:53.233568 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:53.233568 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:53.234478 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d247b1-9960-44ec-bd27-4752ecfe3f53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:53.236254 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d247b1-9960-44ec-bd27-4752ecfe3f53 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:53.240348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 385/792] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:42:53.245918 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c997b84a-6e71-4b0a-8a6b-2817574bfaeb None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:53.269614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 408/793] 10.222.0.17 () {56 vars in 1135 bytes} [Sat Sep 10 23:42:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:53.278098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d247b1-9960-44ec-bd27-4752ecfe3f53 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:53.382376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c997b84a-6e71-4b0a-8a6b-2817574bfaeb None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:53.398611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:53.406014 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d247b1-9960-44ec-bd27-4752ecfe3f53 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:53.417424 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:53.420160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b6912-fcc7-4e39-94cc-ac86651bf73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:53.420989 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b6912-fcc7-4e39-94cc-ac86651bf73a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:53.421459 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b6912-fcc7-4e39-94cc-ac86651bf73a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:53.421920 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b6912-fcc7-4e39-94cc-ac86651bf73a None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:53.422790 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b6912-fcc7-4e39-94cc-ac86651bf73a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:53.423283 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b6912-fcc7-4e39-94cc-ac86651bf73a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:53.424988 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b6912-fcc7-4e39-94cc-ac86651bf73a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:53.437892 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 386/794] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:53.439450 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bf4da-e8d1-4f10-8421-0826f06109df None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:53.439972 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bf4da-e8d1-4f10-8421-0826f06109df None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:53.440439 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bf4da-e8d1-4f10-8421-0826f06109df None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:53.446740 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf52015d-c767-4f8b-bf26-261802b2ac2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:53.446740 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52015d-c767-4f8b-bf26-261802b2ac2f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:53.446740 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52015d-c767-4f8b-bf26-261802b2ac2f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:53.451766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52015d-c767-4f8b-bf26-261802b2ac2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:53.451875 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 387/795] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:53.451875 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383bf4da-e8d1-4f10-8421-0826f06109df None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:53.460016 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf52015d-c767-4f8b-bf26-261802b2ac2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:53.461791 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf52015d-c767-4f8b-bf26-261802b2ac2f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:53.479665 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-cf52015d-c767-4f8b-bf26-261802b2ac2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:53.479665 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:53.479665 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:53.479665 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:53.479665 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:53.479665 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:53.479665 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:53.479665 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:53.479665 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:53.479665 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:53.479665 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:53.479665 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:53.479665 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:53.479665 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:53.479665 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:53.479665 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:53.480660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:53.480660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:53.480660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:53.480660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:53.480660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:53.480660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:53.480660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:53.480660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:53.480660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:53.480660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:53.480660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:53.480660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:53.480660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:53.480660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:53.480660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:53.480660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:53.480660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:53.481553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:53.481553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:53.481553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:53.486012 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 409/796] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:53.487335 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaacd02e-6857-4a08-84e5-c5fe8edff432 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:53.487619 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaacd02e-6857-4a08-84e5-c5fe8edff432 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:53.488006 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaacd02e-6857-4a08-84e5-c5fe8edff432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:53.509705 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-eaacd02e-6857-4a08-84e5-c5fe8edff432 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:53.510977 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-eaacd02e-6857-4a08-84e5-c5fe8edff432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:53.568412 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383bf4da-e8d1-4f10-8421-0826f06109df None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:53.594694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 410/797] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:42:53.594694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaacd02e-6857-4a08-84e5-c5fe8edff432 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:53.648431 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c6ba04-3dc4-4576-ba11-62026067e80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:53.649417 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6ba04-3dc4-4576-ba11-62026067e80d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:53.649909 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6ba04-3dc4-4576-ba11-62026067e80d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:53.650390 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6ba04-3dc4-4576-ba11-62026067e80d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:53.668822 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c6ba04-3dc4-4576-ba11-62026067e80d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:53.670556 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c6ba04-3dc4-4576-ba11-62026067e80d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:53.683590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-78c6ba04-3dc4-4576-ba11-62026067e80d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 23:42:53.683590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:53.683590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:53.683590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:53.683590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:53.683590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:53.683590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:53.683590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:53.683590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:53.683590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:53.683590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:53.683590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:53.683590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:42:53.683590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:42:53.683590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:42:53.683590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:42:53.684591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:53.684591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:53.684591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:42:53.684591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:53.684591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:42:53.684591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:53.684591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:53.684591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:53.684591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:53.684591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:53.684591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:53.684591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:53.684591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:53.684591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:53.684591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:53.684591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:53.684591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:53.684591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:53.685544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:53.685544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:53.685544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:42:53.685544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:42:53.685544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:42:53.685544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:42:53.685544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:42:53.685544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:42:53.685544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 23:42:53.685544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:53.688802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 388/798] 10.222.0.17 () {56 vars in 1093 bytes} [Sat Sep 10 23:42:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:53.693881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c6ba04-3dc4-4576-ba11-62026067e80d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:53.714726 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:53.760251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:53.801434 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaacd02e-6857-4a08-84e5-c5fe8edff432 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:53.813094 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c6ba04-3dc4-4576-ba11-62026067e80d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:53.849470 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4e4b6-ec15-456f-97e8-673a8c34e5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:53.850397 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4e4b6-ec15-456f-97e8-673a8c34e5c9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:53.850916 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4e4b6-ec15-456f-97e8-673a8c34e5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:53.851381 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4e4b6-ec15-456f-97e8-673a8c34e5c9 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:53.851916 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af0c87-cc3f-4fc2-a41b-8ca59d323e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:53.852535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af0c87-cc3f-4fc2-a41b-8ca59d323e87 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:53.852808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af0c87-cc3f-4fc2-a41b-8ca59d323e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:53.853084 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af0c87-cc3f-4fc2-a41b-8ca59d323e87 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:53.853332 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4e4b6-ec15-456f-97e8-673a8c34e5c9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:53.853724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af0c87-cc3f-4fc2-a41b-8ca59d323e87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:53.854026 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af0c87-cc3f-4fc2-a41b-8ca59d323e87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:53.854213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4e4b6-ec15-456f-97e8-673a8c34e5c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:53.855742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af0c87-cc3f-4fc2-a41b-8ca59d323e87 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:53.856081 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4e4b6-ec15-456f-97e8-673a8c34e5c9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:53.874954 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 389/799] 10.222.0.17 () {56 vars in 1133 bytes} [Sat Sep 10 23:42:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:53.876533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 411/800] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:53.882166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af0c87-cc3f-4fc2-a41b-8ca59d323e87 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:53.882431 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4e4b6-ec15-456f-97e8-673a8c34e5c9 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:54.032630 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4e4b6-ec15-456f-97e8-673a8c34e5c9 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:54.045032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:54.048599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af0c87-cc3f-4fc2-a41b-8ca59d323e87 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:54.076222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e30bf7-f31d-4a68-ae6f-f1108af5f543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:54.077862 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e30bf7-f31d-4a68-ae6f-f1108af5f543 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:54.078586 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e30bf7-f31d-4a68-ae6f-f1108af5f543 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:54.079077 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e30bf7-f31d-4a68-ae6f-f1108af5f543 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:54.086348 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13125dd4-e5f0-4f6e-ab9b-f4bb65318bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:54.087000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13125dd4-e5f0-4f6e-ab9b-f4bb65318bac None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:54.087275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13125dd4-e5f0-4f6e-ab9b-f4bb65318bac None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:54.087544 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13125dd4-e5f0-4f6e-ab9b-f4bb65318bac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:54.095224 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13125dd4-e5f0-4f6e-ab9b-f4bb65318bac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:54.096630 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13125dd4-e5f0-4f6e-ab9b-f4bb65318bac None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:54.098719 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:54.126168 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-13125dd4-e5f0-4f6e-ab9b-f4bb65318bac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:54.126168 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:54.126168 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:54.126168 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:54.126168 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:54.126168 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:54.126168 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:54.126168 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:54.126168 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:54.126168 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:54.126168 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:54.126168 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:54.126168 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:42:54.126168 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:42:54.126168 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:42:54.127285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:42:54.127285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:54.127285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:54.127285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:54.127285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:54.127285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:54.127285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:54.127285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:54.127285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:54.127285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:54.127285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:54.127285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:54.127285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:54.127285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:54.127285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:54.127285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:54.127285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:54.128181 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:42:54.128181 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:42:54.128181 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:42:54.128181 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:42:54.128181 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:42:54.128181 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:42:54.128181 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:54.128181 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:54.128181 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 412/801] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:54.128181 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13125dd4-e5f0-4f6e-ab9b-f4bb65318bac None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:54.171838 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e30bf7-f31d-4a68-ae6f-f1108af5f543 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:54.174517 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e30bf7-f31d-4a68-ae6f-f1108af5f543 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:54.232672 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13125dd4-e5f0-4f6e-ab9b-f4bb65318bac None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:54.283332 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715bbe89-bed6-41e1-95bb-b1bd9216ef24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:54.283965 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715bbe89-bed6-41e1-95bb-b1bd9216ef24 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:54.284238 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715bbe89-bed6-41e1-95bb-b1bd9216ef24 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:54.284504 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715bbe89-bed6-41e1-95bb-b1bd9216ef24 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:54.285086 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715bbe89-bed6-41e1-95bb-b1bd9216ef24 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:54.286449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715bbe89-bed6-41e1-95bb-b1bd9216ef24 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:54.297088 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-715bbe89-bed6-41e1-95bb-b1bd9216ef24 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 23:42:54.297088 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:54.297088 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 23:42:54.297088 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 23:42:54.297088 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 23:42:54.297088 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 23:42:54.297088 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 23:42:54.297088 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:42:54.297088 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 23:42:54.297088 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:54.297088 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:42:54.297088 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:54.297088 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:54.297088 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:54.297088 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:54.297088 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:54.297088 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:54.298135 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:54.298135 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:54.298135 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:54.298135 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:54.298135 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:54.298135 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:54.298135 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 23:42:54.298135 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 23:42:54.298135 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 23:42:54.298135 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 23:42:54.298135 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:54.298135 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:54.298135 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:54.298135 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:54.298135 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:54.298135 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:54.298979 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:54.298979 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:54.298979 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:54.298979 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:54.298979 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:54.298979 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:54.298979 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:54.298979 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:54.298979 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:54.298979 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:54.298979 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 23:42:54.298979 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:42:54.298979 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 23:42:54.298979 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:54.304736 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 413/802] 10.222.0.17 () {56 vars in 1129 bytes} [Sat Sep 10 23:42:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:54.305646 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715bbe89-bed6-41e1-95bb-b1bd9216ef24 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:54.321760 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 390/803] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:42:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 23:42:54.323844 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e30bf7-f31d-4a68-ae6f-f1108af5f543 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:54.333904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:54.371572 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:54.426402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715bbe89-bed6-41e1-95bb-b1bd9216ef24 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:54.436180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e30bf7-f31d-4a68-ae6f-f1108af5f543 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:54.465355 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60178c5d-f8f7-4d0a-9f45-9e3d8ea0fb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:54.466089 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60178c5d-f8f7-4d0a-9f45-9e3d8ea0fb14 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:54.466366 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60178c5d-f8f7-4d0a-9f45-9e3d8ea0fb14 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:54.466629 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60178c5d-f8f7-4d0a-9f45-9e3d8ea0fb14 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:54.467468 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60178c5d-f8f7-4d0a-9f45-9e3d8ea0fb14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:54.467763 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60178c5d-f8f7-4d0a-9f45-9e3d8ea0fb14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:54.469226 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60178c5d-f8f7-4d0a-9f45-9e3d8ea0fb14 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:54.477679 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ce92b-bc82-4b34-9dc9-45e90d2f3caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:54.478293 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ce92b-bc82-4b34-9dc9-45e90d2f3caf None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:54.478568 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ce92b-bc82-4b34-9dc9-45e90d2f3caf None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:54.478866 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ce92b-bc82-4b34-9dc9-45e90d2f3caf None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:54.479503 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ce92b-bc82-4b34-9dc9-45e90d2f3caf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:54.479831 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ce92b-bc82-4b34-9dc9-45e90d2f3caf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:54.481214 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ce92b-bc82-4b34-9dc9-45e90d2f3caf None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:54.490343 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 391/804] 10.222.0.17 () {56 vars in 1124 bytes} [Sat Sep 10 23:42:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:54.490495 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 414/805] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:42:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:54.490597 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2542d3d-1cff-4d74-bb77-002b88ab9d4b None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:54.490597 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2542d3d-1cff-4d74-bb77-002b88ab9d4b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:54.490597 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2542d3d-1cff-4d74-bb77-002b88ab9d4b None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:54.490825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60178c5d-f8f7-4d0a-9f45-9e3d8ea0fb14 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:54.490898 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 392/806] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:54.490898 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2542d3d-1cff-4d74-bb77-002b88ab9d4b None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:54.567953 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:54.595512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60178c5d-f8f7-4d0a-9f45-9e3d8ea0fb14 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:54.600778 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2542d3d-1cff-4d74-bb77-002b88ab9d4b None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:54.617559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:54.628643 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f376dd-7381-49c4-ae59-59e28d6eefea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:54.629261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f376dd-7381-49c4-ae59-59e28d6eefea None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:54.629530 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f376dd-7381-49c4-ae59-59e28d6eefea None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:54.629795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f376dd-7381-49c4-ae59-59e28d6eefea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:54.634509 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84e20d-986b-40cc-81b3-426fac10afb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:54.635180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84e20d-986b-40cc-81b3-426fac10afb6 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:54.635453 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84e20d-986b-40cc-81b3-426fac10afb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:54.635720 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84e20d-986b-40cc-81b3-426fac10afb6 None admin] PATH_INFO: `/v3/projects/947277c4458645899287e6818947c076/users/564e65585e9543e286b6041af870cf41/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:54.638446 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a84e20d-986b-40cc-81b3-426fac10afb6 None admin] RBAC: Authorizing `identity:create_grant(project_id=947277c4458645899287e6818947c076, user_id=564e65585e9543e286b6041af870cf41, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:54.639901 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a84e20d-986b-40cc-81b3-426fac10afb6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:54.640075 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f376dd-7381-49c4-ae59-59e28d6eefea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:54.641469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f376dd-7381-49c4-ae59-59e28d6eefea None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:54.656469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-e9f376dd-7381-49c4-ae59-59e28d6eefea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:54.656469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:54.656469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:54.656469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:54.656469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:54.656469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:54.656469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:54.656469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:54.656469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:54.656469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:54.656469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:54.656469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:54.656469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:54.656469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:54.656469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:54.656469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:54.657599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:54.657599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:54.657599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:54.657599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:54.657599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:54.657599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:54.657599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:54.657599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:54.657599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:54.657599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:54.657599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:54.657599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:54.657599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:54.657599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:54.657599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:54.657599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:54.657599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:54.658566 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:54.658566 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:54.658566 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:54.658566 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 415/807] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:54.658839 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f376dd-7381-49c4-ae59-59e28d6eefea None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:54.734480 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 393/808] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Sep 10 23:42:54 2022] PUT /identity/v3/projects/947277c4458645899287e6818947c076/users/564e65585e9543e286b6041af870cf41/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:42:54.735679 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff57abc-9d20-4693-8275-6f3ac9eb3e88 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:54.735958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff57abc-9d20-4693-8275-6f3ac9eb3e88 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:54.736227 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff57abc-9d20-4693-8275-6f3ac9eb3e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:54.758905 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-fff57abc-9d20-4693-8275-6f3ac9eb3e88 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:54.760142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fff57abc-9d20-4693-8275-6f3ac9eb3e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:54.797400 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f376dd-7381-49c4-ae59-59e28d6eefea None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:54.843823 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 394/809] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:42:54.845344 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788329c-a9f8-4f01-abf7-e8eaefd846fe None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:54.845623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788329c-a9f8-4f01-abf7-e8eaefd846fe None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:54.845888 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788329c-a9f8-4f01-abf7-e8eaefd846fe None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:54.847359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 395/810] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:54.847892 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788329c-a9f8-4f01-abf7-e8eaefd846fe None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:54.856897 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad81070-510e-4413-9056-6c7bd080171c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:54.857963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad81070-510e-4413-9056-6c7bd080171c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:54.858422 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad81070-510e-4413-9056-6c7bd080171c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:54.858916 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad81070-510e-4413-9056-6c7bd080171c None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:54.859673 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad81070-510e-4413-9056-6c7bd080171c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:54.860050 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad81070-510e-4413-9056-6c7bd080171c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:54.861521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad81070-510e-4413-9056-6c7bd080171c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:54.872488 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 416/811] 10.222.0.17 () {56 vars in 1124 bytes} [Sat Sep 10 23:42:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:54.872488 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67829f87-9372-41f4-a05e-1127b9c28072 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:54.872488 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67829f87-9372-41f4-a05e-1127b9c28072 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:54.872488 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67829f87-9372-41f4-a05e-1127b9c28072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:54.890167 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-67829f87-9372-41f4-a05e-1127b9c28072 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:54.891515 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-67829f87-9372-41f4-a05e-1127b9c28072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:54.938290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:54.943473 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:54.986803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 417/812] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:42:54.987836 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67829f87-9372-41f4-a05e-1127b9c28072 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:55.000804 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788329c-a9f8-4f01-abf7-e8eaefd846fe None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:55.080217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f8572a-1d12-4659-a947-b8276f97f2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:55.080973 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8572a-1d12-4659-a947-b8276f97f2c9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:55.081318 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8572a-1d12-4659-a947-b8276f97f2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:55.081657 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8572a-1d12-4659-a947-b8276f97f2c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:55.082426 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f8572a-1d12-4659-a947-b8276f97f2c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:55.082821 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f8572a-1d12-4659-a947-b8276f97f2c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:55.084264 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f8572a-1d12-4659-a947-b8276f97f2c9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:55.103524 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 396/813] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:55.109161 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f8572a-1d12-4659-a947-b8276f97f2c9 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:55.182699 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67829f87-9372-41f4-a05e-1127b9c28072 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:55.226595 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d7ee1-605f-4a4b-8e78-eab6f6f9ed2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:55.227586 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d7ee1-605f-4a4b-8e78-eab6f6f9ed2e None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:55.227924 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d7ee1-605f-4a4b-8e78-eab6f6f9ed2e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:55.228358 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d7ee1-605f-4a4b-8e78-eab6f6f9ed2e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:55.229186 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9d7ee1-605f-4a4b-8e78-eab6f6f9ed2e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:55.231607 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9d7ee1-605f-4a4b-8e78-eab6f6f9ed2e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:55.238465 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f8572a-1d12-4659-a947-b8276f97f2c9 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:55.262721 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:55.277650 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb9a29-44e0-470b-8107-045eaa305812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:55.278267 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb9a29-44e0-470b-8107-045eaa305812 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:55.278538 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb9a29-44e0-470b-8107-045eaa305812 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:55.278894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb9a29-44e0-470b-8107-045eaa305812 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:55.285704 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfb9a29-44e0-470b-8107-045eaa305812 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:55.287167 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfb9a29-44e0-470b-8107-045eaa305812 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:55.298896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-9dfb9a29-44e0-470b-8107-045eaa305812 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:55.298896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:55.298896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:55.298896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:55.298896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:55.298896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:55.298896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:55.298896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:55.298896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:55.298896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:55.298896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:55.298896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:55.298896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:42:55.298896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:42:55.298896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:42:55.305528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:42:55.305528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:55.305528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:55.305528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:55.305528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:55.305528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:55.305528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:55.305528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:55.305528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:55.305528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:55.305528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:55.305528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:55.305528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:55.305528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:55.305528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:55.305528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:55.305528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:55.306441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:42:55.306441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:42:55.306441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:42:55.306441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:42:55.306441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:42:55.306441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:42:55.306441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:55.306441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:55.306441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 397/814] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:55.306441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb9a29-44e0-470b-8107-045eaa305812 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:55.362833 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 418/815] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Sep 10 23:42:54 2022] POST /identity/v3/endpoints => generated 346 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:42:55.372053 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d7ee1-605f-4a4b-8e78-eab6f6f9ed2e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:55.408210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:55.424376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb9a29-44e0-470b-8107-045eaa305812 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:55.477059 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac088624-b194-4df4-8fd4-9d5a7a1d64a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:55.477695 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac088624-b194-4df4-8fd4-9d5a7a1d64a8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:55.477969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac088624-b194-4df4-8fd4-9d5a7a1d64a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:55.478235 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac088624-b194-4df4-8fd4-9d5a7a1d64a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:55.499757 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac088624-b194-4df4-8fd4-9d5a7a1d64a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:55.505929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-ac088624-b194-4df4-8fd4-9d5a7a1d64a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:55.505929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:55.505929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:55.505929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:55.505929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:55.505929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:55.505929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:55.505929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:55.505929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:55.505929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:55.505929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:55.505929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:55.505929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:55.505929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:55.505929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:55.505929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:55.506926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:55.506926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:55.506926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:55.506926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:55.506926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:55.506926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:55.506926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:55.506926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:55.506926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:55.506926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:55.506926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:55.506926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:55.506926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:55.506926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:55.506926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:55.506926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:55.507763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:55.507763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:55.507763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:55.507763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:55.507763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:55.507763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:55.507763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:55.507763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:55.507763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:55.507763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:55.507763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:42:55.507763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:55.512862 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 398/816] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Sep 10 23:42:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:55.514033 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac088624-b194-4df4-8fd4-9d5a7a1d64a8 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:55.526422 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d7ee1-605f-4a4b-8e78-eab6f6f9ed2e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:55.553806 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:55.575166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b3ff8-a2cb-4555-8603-8f2679bf9ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:55.576149 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b3ff8-a2cb-4555-8603-8f2679bf9ef9 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:55.576427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b3ff8-a2cb-4555-8603-8f2679bf9ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:55.576691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b3ff8-a2cb-4555-8603-8f2679bf9ef9 None admin] PATH_INFO: `/v3/services/network` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:55.577271 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b3ff8-a2cb-4555-8603-8f2679bf9ef9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:55.578627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b3ff8-a2cb-4555-8603-8f2679bf9ef9 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:55.588531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-1e6b3ff8-a2cb-4555-8603-8f2679bf9ef9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 23:42:55.588531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:55.588531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 23:42:55.588531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 23:42:55.588531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 23:42:55.588531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 23:42:55.588531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 23:42:55.588531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:42:55.588531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 23:42:55.588531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:55.588531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:42:55.588531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:55.588531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:55.588531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:55.588531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:55.588531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:55.588531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:55.589576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:55.589576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:55.589576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:55.589576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:55.589576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:55.589576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:55.589576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 23:42:55.589576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 23:42:55.589576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 23:42:55.589576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 23:42:55.589576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:55.589576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:55.589576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:55.589576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:55.589576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:55.589576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:55.590537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:55.590537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:55.590537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:55.590537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:55.590537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:55.590537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:55.590537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:55.590537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:55.590537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:55.590537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:55.590537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 23:42:55.590537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:42:55.590537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 23:42:55.590537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:55.590537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 419/817] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:55 2022] GET /identity/v3/services/network => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:55.591854 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b3ff8-a2cb-4555-8603-8f2679bf9ef9 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:55.646370 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac088624-b194-4df4-8fd4-9d5a7a1d64a8 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:55.692395 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd7b9c-1320-4062-a9b8-18def6e0f19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:55.692994 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd7b9c-1320-4062-a9b8-18def6e0f19a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:55.693415 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd7b9c-1320-4062-a9b8-18def6e0f19a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:55.693686 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd7b9c-1320-4062-a9b8-18def6e0f19a None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:55.694708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd7b9c-1320-4062-a9b8-18def6e0f19a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:55.695095 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd7b9c-1320-4062-a9b8-18def6e0f19a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:55.696587 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd7b9c-1320-4062-a9b8-18def6e0f19a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:55.699591 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:55.711084 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b3ff8-a2cb-4555-8603-8f2679bf9ef9 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:55.712346 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 399/818] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:42:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:55.714921 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd7b9c-1320-4062-a9b8-18def6e0f19a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:55.757156 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814b7d9-511e-4bac-a6d7-bdb008f1d9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:55.757891 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814b7d9-511e-4bac-a6d7-bdb008f1d9a4 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:55.758284 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814b7d9-511e-4bac-a6d7-bdb008f1d9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:55.758650 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814b7d9-511e-4bac-a6d7-bdb008f1d9a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:55.759450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814b7d9-511e-4bac-a6d7-bdb008f1d9a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:55.759849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814b7d9-511e-4bac-a6d7-bdb008f1d9a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:55.762593 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814b7d9-511e-4bac-a6d7-bdb008f1d9a4 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:55.780587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 420/819] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Sep 10 23:42:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:55.781672 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814b7d9-511e-4bac-a6d7-bdb008f1d9a4 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:55.836341 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:55.842258 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd7b9c-1320-4062-a9b8-18def6e0f19a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:55.888788 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e6c74-d648-46d8-9180-0393cdf5edf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:55.889551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e6c74-d648-46d8-9180-0393cdf5edf0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:55.889948 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e6c74-d648-46d8-9180-0393cdf5edf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:55.890318 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e6c74-d648-46d8-9180-0393cdf5edf0 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:55.891066 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e6c74-d648-46d8-9180-0393cdf5edf0 None admin] RBAC: Adding query filter params (name=network) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:55.891461 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e6c74-d648-46d8-9180-0393cdf5edf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:55.892916 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e6c74-d648-46d8-9180-0393cdf5edf0 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:55.898039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 400/820] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Sep 10 23:42:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:55.899124 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e6c74-d648-46d8-9180-0393cdf5edf0 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:55.926480 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814b7d9-511e-4bac-a6d7-bdb008f1d9a4 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:55.975188 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74840b1d-eb8d-40ca-ab25-1ae09c2dbabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:55.976035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74840b1d-eb8d-40ca-ab25-1ae09c2dbabc None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:55.976315 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74840b1d-eb8d-40ca-ab25-1ae09c2dbabc None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:55.976583 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74840b1d-eb8d-40ca-ab25-1ae09c2dbabc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:55.978194 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74840b1d-eb8d-40ca-ab25-1ae09c2dbabc None admin] RBAC: Adding query filter params (role.id=9dce48a2b53c482d931dcd064b07744e, scope.project.id=947277c4458645899287e6818947c076, user.id=636bfafb693044d283530063047fc0c2) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:55.978502 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74840b1d-eb8d-40ca-ab25-1ae09c2dbabc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:55.980049 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74840b1d-eb8d-40ca-ab25-1ae09c2dbabc None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:55.987457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 421/821] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:42:55 2022] GET /identity/v3/role_assignments?user.id=636bfafb693044d283530063047fc0c2&scope.project.id=947277c4458645899287e6818947c076&role.id=9dce48a2b53c482d931dcd064b07744e => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:55.991597 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74840b1d-eb8d-40ca-ab25-1ae09c2dbabc None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:56.021697 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:56.029825 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e6c74-d648-46d8-9180-0393cdf5edf0 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:56.074639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cea5e-b430-4bbe-9fd7-b4c4da8c21f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:56.075295 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cea5e-b430-4bbe-9fd7-b4c4da8c21f9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:56.075572 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cea5e-b430-4bbe-9fd7-b4c4da8c21f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:56.075839 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cea5e-b430-4bbe-9fd7-b4c4da8c21f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:56.089556 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78cea5e-b430-4bbe-9fd7-b4c4da8c21f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:56.091109 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78cea5e-b430-4bbe-9fd7-b4c4da8c21f9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:56.109400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-d78cea5e-b430-4bbe-9fd7-b4c4da8c21f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:56.109400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:56.109400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:56.109400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:56.109400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:56.109400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:56.109400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:56.109400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:56.109400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:56.109400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:56.109400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:56.109400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:56.109400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:56.109400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:56.109400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:56.109400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:56.111669 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:56.111669 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:56.111669 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:56.111669 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:56.111669 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:56.111669 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:56.111669 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:56.111669 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:56.111669 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:56.111669 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:56.111669 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:56.111669 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:56.111669 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:56.111669 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:56.111669 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:56.111669 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:56.111669 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:56.113829 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:56.113829 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:56.113829 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:56.118928 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 401/822] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:56.119909 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cea5e-b430-4bbe-9fd7-b4c4da8c21f9 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:56.138894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:56.208918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74840b1d-eb8d-40ca-ab25-1ae09c2dbabc None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:56.238816 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cea5e-b430-4bbe-9fd7-b4c4da8c21f9 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:56.278192 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb32f34-09e1-44d6-a89f-fe978b0b046c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:56.278848 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb32f34-09e1-44d6-a89f-fe978b0b046c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:56.279123 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb32f34-09e1-44d6-a89f-fe978b0b046c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:56.279384 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb32f34-09e1-44d6-a89f-fe978b0b046c None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:56.280021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb32f34-09e1-44d6-a89f-fe978b0b046c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:56.280315 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb32f34-09e1-44d6-a89f-fe978b0b046c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:56.281680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb32f34-09e1-44d6-a89f-fe978b0b046c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:56.296182 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 402/823] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:42:56.304409 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb32f34-09e1-44d6-a89f-fe978b0b046c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:56.311311 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566cbfb8-81bd-43e7-ada3-3d51974d3806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:56.311774 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566cbfb8-81bd-43e7-ada3-3d51974d3806 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:56.312229 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566cbfb8-81bd-43e7-ada3-3d51974d3806 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:56.312668 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566cbfb8-81bd-43e7-ada3-3d51974d3806 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:56.315042 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566cbfb8-81bd-43e7-ada3-3d51974d3806 None admin] RBAC: Adding query filter params (type=network) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:56.315153 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566cbfb8-81bd-43e7-ada3-3d51974d3806 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:56.316507 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566cbfb8-81bd-43e7-ada3-3d51974d3806 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:56.323285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 422/824] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Sep 10 23:42:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:56.328787 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566cbfb8-81bd-43e7-ada3-3d51974d3806 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:56.406556 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:56.436611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:56.441763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb32f34-09e1-44d6-a89f-fe978b0b046c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:56.462957 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566cbfb8-81bd-43e7-ada3-3d51974d3806 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:56.480537 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82627c-a344-4a3e-8fc5-2d0bfb5bc89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:56.481231 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82627c-a344-4a3e-8fc5-2d0bfb5bc89d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:56.481503 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82627c-a344-4a3e-8fc5-2d0bfb5bc89d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:56.481770 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82627c-a344-4a3e-8fc5-2d0bfb5bc89d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:56.490473 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d82627c-a344-4a3e-8fc5-2d0bfb5bc89d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:56.492979 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d82627c-a344-4a3e-8fc5-2d0bfb5bc89d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:56.504857 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae6ba6-756e-4974-b39a-cc1a69e2e37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:56.505838 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae6ba6-756e-4974-b39a-cc1a69e2e37b None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:56.506139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-5d82627c-a344-4a3e-8fc5-2d0bfb5bc89d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 23:42:56.506139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:56.506139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:56.506139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:56.506139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:56.506139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:56.506139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:56.506139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:56.506139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:56.506139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:56.506139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:56.506139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:56.506139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:42:56.506139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:42:56.506139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:42:56.506139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:42:56.507061 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:56.507061 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:56.507061 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:42:56.507061 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:56.507061 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:42:56.507061 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:56.507061 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:56.507061 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:56.507061 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:56.507061 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:56.507061 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:56.507061 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:56.507061 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:56.507061 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:56.507061 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:56.507061 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:56.507061 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:56.507061 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:56.508008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:56.508008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:56.508008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:42:56.508008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:42:56.508008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:42:56.508008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:42:56.508008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:42:56.508008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:42:56.508008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 23:42:56.508008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:56.509389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae6ba6-756e-4974-b39a-cc1a69e2e37b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:56.509851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae6ba6-756e-4974-b39a-cc1a69e2e37b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:56.510756 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae6ba6-756e-4974-b39a-cc1a69e2e37b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:56.513367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae6ba6-756e-4974-b39a-cc1a69e2e37b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:56.514439 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 403/825] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Sep 10 23:42:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:56.514913 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82627c-a344-4a3e-8fc5-2d0bfb5bc89d None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:56.578001 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 423/826] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Sep 10 23:42:56 2022] POST /identity/v3/endpoints => generated 337 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:42:56.641028 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82627c-a344-4a3e-8fc5-2d0bfb5bc89d None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:56.689091 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:56.705793 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d6640-8b3b-478d-b174-da013bb5e60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:56.706425 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d6640-8b3b-478d-b174-da013bb5e60f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:56.706730 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d6640-8b3b-478d-b174-da013bb5e60f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:56.707005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d6640-8b3b-478d-b174-da013bb5e60f None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:56.707754 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d6640-8b3b-478d-b174-da013bb5e60f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:56.708061 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d6640-8b3b-478d-b174-da013bb5e60f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:56.709526 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d6640-8b3b-478d-b174-da013bb5e60f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:56.738565 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 404/827] 10.222.0.17 () {56 vars in 1149 bytes} [Sat Sep 10 23:42:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:56.745664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae6ba6-756e-4974-b39a-cc1a69e2e37b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:56.904251 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae6ba6-756e-4974-b39a-cc1a69e2e37b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:56.944161 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20568733-a2dc-4215-a1d6-add634a1a53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:56.944797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20568733-a2dc-4215-a1d6-add634a1a53d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:56.945068 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20568733-a2dc-4215-a1d6-add634a1a53d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:56.945332 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20568733-a2dc-4215-a1d6-add634a1a53d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:56.947492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:56.958107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20568733-a2dc-4215-a1d6-add634a1a53d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:56.959593 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20568733-a2dc-4215-a1d6-add634a1a53d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:56.975404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-20568733-a2dc-4215-a1d6-add634a1a53d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:56.975404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:56.975404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:56.975404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:56.975404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:56.975404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:56.975404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:56.975404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:56.975404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:56.975404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:56.975404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:56.975404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:56.975404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:56.975404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:56.975404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:56.975404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:56.977953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:56.977953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:56.977953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:56.977953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:56.977953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:56.977953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:56.977953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:56.977953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:56.977953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:56.977953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:56.977953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:56.977953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:56.977953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:56.977953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:56.977953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:56.977953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:56.977953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:56.979179 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:56.979179 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:56.979179 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:56.979179 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 424/828] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:56.982742 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d6640-8b3b-478d-b174-da013bb5e60f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:57.041493 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25edcb0-602e-4462-b44d-e26d3d24812f None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:57.042029 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25edcb0-602e-4462-b44d-e26d3d24812f None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:57.042368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25edcb0-602e-4462-b44d-e26d3d24812f None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:57.044382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 425/829] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:57.049603 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a98d8-2984-4822-a20a-6217a615f029 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:57.049877 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a98d8-2984-4822-a20a-6217a615f029 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:57.050146 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a98d8-2984-4822-a20a-6217a615f029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:57.071009 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-283a98d8-2984-4822-a20a-6217a615f029 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:57.072204 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-283a98d8-2984-4822-a20a-6217a615f029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:57.114334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d6640-8b3b-478d-b174-da013bb5e60f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:57.149395 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 426/830] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:42:57.168522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de065388-1994-4aaa-b81b-b86c2a663609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:57.169366 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de065388-1994-4aaa-b81b-b86c2a663609 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:57.169791 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de065388-1994-4aaa-b81b-b86c2a663609 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:57.170206 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de065388-1994-4aaa-b81b-b86c2a663609 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:57.171011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de065388-1994-4aaa-b81b-b86c2a663609 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:57.171454 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de065388-1994-4aaa-b81b-b86c2a663609 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:57.172995 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de065388-1994-4aaa-b81b-b86c2a663609 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:57.186947 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 405/831] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:57.193754 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a98d8-2984-4822-a20a-6217a615f029 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:57.304863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a98d8-2984-4822-a20a-6217a615f029 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:57.328036 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:57.343037 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d93ca02-10f9-4975-a706-fede9fbdc774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:57.343669 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93ca02-10f9-4975-a706-fede9fbdc774 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:57.343946 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93ca02-10f9-4975-a706-fede9fbdc774 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:57.344213 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93ca02-10f9-4975-a706-fede9fbdc774 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:57.351239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d93ca02-10f9-4975-a706-fede9fbdc774 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:57.355173 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d93ca02-10f9-4975-a706-fede9fbdc774 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:57.366069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-9d93ca02-10f9-4975-a706-fede9fbdc774 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:57.366069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:57.366069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:57.366069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:57.366069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:57.366069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:57.366069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:57.366069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:57.366069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:57.366069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:57.366069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:57.366069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:57.366069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:42:57.366069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:42:57.366069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:42:57.367210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:42:57.367210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:57.367210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:57.367210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:57.367210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:57.367210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:57.367210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:57.367210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:57.367210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:57.367210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:57.367210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:57.367210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:57.367210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:57.367210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:57.367210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:57.367210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:57.367210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:57.368093 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:42:57.368093 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:42:57.368093 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:42:57.368093 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:42:57.368093 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:42:57.368093 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:42:57.368093 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:57.368093 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:57.369281 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 427/832] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:57.374226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de065388-1994-4aaa-b81b-b86c2a663609 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:57.428615 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:57.446111 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d93ca02-10f9-4975-a706-fede9fbdc774 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:57.505146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de065388-1994-4aaa-b81b-b86c2a663609 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:57.551974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71acda-f453-489e-aadf-dac249d7dad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:57.552819 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71acda-f453-489e-aadf-dac249d7dad0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:57.553508 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71acda-f453-489e-aadf-dac249d7dad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:57.553789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71acda-f453-489e-aadf-dac249d7dad0 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:57.554783 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71acda-f453-489e-aadf-dac249d7dad0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:57.555092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71acda-f453-489e-aadf-dac249d7dad0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:57.557477 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71acda-f453-489e-aadf-dac249d7dad0 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:57.562866 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d93ca02-10f9-4975-a706-fede9fbdc774 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:57.574573 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 406/833] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:42:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:57.579974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71acda-f453-489e-aadf-dac249d7dad0 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:57.603514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45409a0-a7e4-4ecc-8e29-42853b514456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:57.604138 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45409a0-a7e4-4ecc-8e29-42853b514456 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:57.604411 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45409a0-a7e4-4ecc-8e29-42853b514456 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:57.604742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45409a0-a7e4-4ecc-8e29-42853b514456 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:57.616700 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45409a0-a7e4-4ecc-8e29-42853b514456 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:42:57.622533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-d45409a0-a7e4-4ecc-8e29-42853b514456 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:42:57.622533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:57.622533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:57.622533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:57.622533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:57.622533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:57.622533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:57.622533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:57.622533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:57.622533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:57.622533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:57.622533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:57.622533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:42:57.622533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:42:57.622533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:42:57.622533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:42:57.623464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:42:57.623464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:42:57.623464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:42:57.623464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:42:57.623464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:57.623464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:57.623464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:57.623464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:57.623464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:57.623464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:57.623464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:57.623464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:57.623464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:57.623464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:57.623464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:57.623464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:57.624581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:57.624581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:57.624581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:57.624581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:57.624581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:42:57.624581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:42:57.624581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:42:57.624581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:42:57.624581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:42:57.624581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:42:57.624581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:42:57.624581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:57.624581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 428/834] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:42:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:42:57.629533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45409a0-a7e4-4ecc-8e29-42853b514456 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:57.680533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:57.689719 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71acda-f453-489e-aadf-dac249d7dad0 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:57.697952 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:57.741467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f16344-66f7-495e-92f3-441582348255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:57.742794 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f16344-66f7-495e-92f3-441582348255 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:57.743262 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f16344-66f7-495e-92f3-441582348255 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:57.743698 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f16344-66f7-495e-92f3-441582348255 None admin] PATH_INFO: `/v3/projects/947277c4458645899287e6818947c076/users/b0a879085a194b0687e1cdcbafd48033/roles/8ff1727b65104e5fb97333d477196d46` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:57.764189 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f16344-66f7-495e-92f3-441582348255 None admin] RBAC: Authorizing `identity:create_grant(project_id=947277c4458645899287e6818947c076, user_id=b0a879085a194b0687e1cdcbafd48033, role_id=8ff1727b65104e5fb97333d477196d46)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:57.765584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f16344-66f7-495e-92f3-441582348255 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:57.777968 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45409a0-a7e4-4ecc-8e29-42853b514456 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:57.814999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e2374-6687-496b-9f29-c1aadb95bc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:57.815606 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e2374-6687-496b-9f29-c1aadb95bc61 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:57.815933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e2374-6687-496b-9f29-c1aadb95bc61 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:57.816208 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e2374-6687-496b-9f29-c1aadb95bc61 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:57.816867 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e2374-6687-496b-9f29-c1aadb95bc61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:57.817167 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e2374-6687-496b-9f29-c1aadb95bc61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:57.818524 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e2374-6687-496b-9f29-c1aadb95bc61 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:57.832967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 429/836] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:42:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:42:57.833423 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 407/836] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Sep 10 23:42:57 2022] PUT /identity/v3/projects/947277c4458645899287e6818947c076/users/b0a879085a194b0687e1cdcbafd48033/roles/8ff1727b65104e5fb97333d477196d46 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:42:57.834630 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e2374-6687-496b-9f29-c1aadb95bc61 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:57.877157 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb8416-8857-41fb-958e-8a385fcb62bc None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:57.877874 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb8416-8857-41fb-958e-8a385fcb62bc None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:57.878335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb8416-8857-41fb-958e-8a385fcb62bc None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:57.880614 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 408/837] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:57.889236 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428627f-a8c1-4416-b742-69bd0c56cc00 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:57.889993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428627f-a8c1-4416-b742-69bd0c56cc00 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:57.891112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428627f-a8c1-4416-b742-69bd0c56cc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:57.915766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c428627f-a8c1-4416-b742-69bd0c56cc00 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:57.918967 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c428627f-a8c1-4416-b742-69bd0c56cc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:57.955077 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:57.960659 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e2374-6687-496b-9f29-c1aadb95bc61 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:58.011967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49dc58b-8e70-47f8-9661-d8f251a66477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:58.012629 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49dc58b-8e70-47f8-9661-d8f251a66477 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:58.012906 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49dc58b-8e70-47f8-9661-d8f251a66477 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:58.013176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49dc58b-8e70-47f8-9661-d8f251a66477 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:58.016140 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 409/838] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 23:42:58.025676 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49dc58b-8e70-47f8-9661-d8f251a66477 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:58.027817 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49dc58b-8e70-47f8-9661-d8f251a66477 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:58.042918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-f49dc58b-8e70-47f8-9661-d8f251a66477 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:58.042918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:58.042918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:58.042918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:58.042918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:58.042918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:58.042918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:58.042918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:58.042918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:58.042918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:58.042918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:58.042918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:58.042918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:58.042918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:58.042918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:58.042918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:58.043963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:58.043963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:58.043963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:58.043963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:58.043963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:58.043963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:58.043963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:58.043963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:58.043963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:58.043963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:58.043963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:58.043963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:58.043963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:58.043963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:58.043963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:58.043963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:58.043963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:58.044857 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:58.044857 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:58.044857 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:58.044857 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 430/839] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:58.050132 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428627f-a8c1-4416-b742-69bd0c56cc00 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:58.139667 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:58.155005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428627f-a8c1-4416-b742-69bd0c56cc00 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:58.196024 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2874cf5-1ffa-42e7-85f2-16dad58cffc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:58.197002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2874cf5-1ffa-42e7-85f2-16dad58cffc7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:58.197509 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2874cf5-1ffa-42e7-85f2-16dad58cffc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:58.197981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2874cf5-1ffa-42e7-85f2-16dad58cffc7 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:58.198969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2874cf5-1ffa-42e7-85f2-16dad58cffc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:58.199474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2874cf5-1ffa-42e7-85f2-16dad58cffc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:58.201040 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2874cf5-1ffa-42e7-85f2-16dad58cffc7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:58.214446 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 410/840] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:58.220342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49dc58b-8e70-47f8-9661-d8f251a66477 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:58.294166 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2874cf5-1ffa-42e7-85f2-16dad58cffc7 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:58.331311 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49dc58b-8e70-47f8-9661-d8f251a66477 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:58.367825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ccd99-1f0a-4114-9f76-d8eeb0f68d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:58.368383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ccd99-1f0a-4114-9f76-d8eeb0f68d22 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:58.368658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ccd99-1f0a-4114-9f76-d8eeb0f68d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:58.368930 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ccd99-1f0a-4114-9f76-d8eeb0f68d22 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:58.377650 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ccd99-1f0a-4114-9f76-d8eeb0f68d22 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:58.379147 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ccd99-1f0a-4114-9f76-d8eeb0f68d22 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:58.392660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-4c2ccd99-1f0a-4114-9f76-d8eeb0f68d22 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 10 23:42:58.392660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:58.392660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:58.392660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:58.392660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:58.392660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:58.392660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:58.392660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:58.392660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:58.392660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:58.392660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:58.392660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:58.392660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:42:58.392660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:42:58.392660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:42:58.392660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:42:58.393636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:58.393636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:58.393636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:42:58.393636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:58.393636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:42:58.393636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:42:58.393636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:58.393636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:58.393636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:58.393636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:58.393636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:58.393636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:58.393636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:58.393636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:58.393636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:58.393636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:58.393636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:58.393636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:58.394655 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:58.394655 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:58.394655 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:42:58.394655 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:42:58.394655 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:42:58.394655 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:42:58.394655 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:42:58.394655 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:42:58.394655 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 10 23:42:58.394655 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:58.398511 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 431/841] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:42:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:58.399407 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ccd99-1f0a-4114-9f76-d8eeb0f68d22 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:58.408334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2874cf5-1ffa-42e7-85f2-16dad58cffc7 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:58.414452 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:58.447754 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6efddd-1f25-4ca9-90c9-b934e7a3b877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:58.448689 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6efddd-1f25-4ca9-90c9-b934e7a3b877 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:58.449116 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6efddd-1f25-4ca9-90c9-b934e7a3b877 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:58.449555 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6efddd-1f25-4ca9-90c9-b934e7a3b877 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:58.450293 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6efddd-1f25-4ca9-90c9-b934e7a3b877 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:58.451841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6efddd-1f25-4ca9-90c9-b934e7a3b877 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:58.461180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-3f6efddd-1f25-4ca9-90c9-b934e7a3b877 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 23:42:58.461180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:58.461180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 23:42:58.461180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 23:42:58.461180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 23:42:58.461180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 23:42:58.461180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 23:42:58.461180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:42:58.461180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 23:42:58.461180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:58.461180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:42:58.461180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:58.461180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:58.461180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:58.461180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:58.461180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:58.461180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:58.462398 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:58.462398 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:58.462398 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:58.462398 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:58.462398 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:58.462398 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:58.462398 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 23:42:58.462398 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 23:42:58.462398 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 23:42:58.462398 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 23:42:58.462398 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:58.462398 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:58.462398 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:58.462398 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:58.462398 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:58.462398 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:58.463430 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:58.463430 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:58.463430 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:58.463430 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:58.463430 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:58.463430 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:58.463430 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:58.463430 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:58.463430 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:58.463430 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:58.463430 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 23:42:58.463430 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:42:58.463430 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 23:42:58.463430 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:58.466100 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 411/842] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Sep 10 23:42:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:42:58.474704 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6efddd-1f25-4ca9-90c9-b934e7a3b877 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:58.485032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:58.516456 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ccd99-1f0a-4114-9f76-d8eeb0f68d22 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:58.553831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8132284-0e9a-49fa-89e0-e6ef92795f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:58.554781 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8132284-0e9a-49fa-89e0-e6ef92795f84 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:58.555286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8132284-0e9a-49fa-89e0-e6ef92795f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:58.555724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8132284-0e9a-49fa-89e0-e6ef92795f84 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:58.556676 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8132284-0e9a-49fa-89e0-e6ef92795f84 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:58.557426 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8132284-0e9a-49fa-89e0-e6ef92795f84 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:58.559896 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8132284-0e9a-49fa-89e0-e6ef92795f84 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:58.578753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 432/843] 10.222.0.17 () {56 vars in 1135 bytes} [Sat Sep 10 23:42:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:58.585137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8132284-0e9a-49fa-89e0-e6ef92795f84 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:58.598482 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6efddd-1f25-4ca9-90c9-b934e7a3b877 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:58.637307 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339e2dc-35d0-4e8a-895d-e50c7bd5db18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:58.637958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339e2dc-35d0-4e8a-895d-e50c7bd5db18 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:58.638231 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339e2dc-35d0-4e8a-895d-e50c7bd5db18 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:58.638495 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339e2dc-35d0-4e8a-895d-e50c7bd5db18 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:58.639166 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2339e2dc-35d0-4e8a-895d-e50c7bd5db18 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:58.639459 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2339e2dc-35d0-4e8a-895d-e50c7bd5db18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:58.640832 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2339e2dc-35d0-4e8a-895d-e50c7bd5db18 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:58.650346 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 412/844] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Sep 10 23:42:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:58.652872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339e2dc-35d0-4e8a-895d-e50c7bd5db18 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:58.679793 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:58.728656 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8132284-0e9a-49fa-89e0-e6ef92795f84 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:58.752501 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:58.781851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b6517-b83c-47c6-9b59-71371612f755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:58.782455 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b6517-b83c-47c6-9b59-71371612f755 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:58.782771 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b6517-b83c-47c6-9b59-71371612f755 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:58.783046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b6517-b83c-47c6-9b59-71371612f755 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:58.785497 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339e2dc-35d0-4e8a-895d-e50c7bd5db18 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:58.838897 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8fde63-23ca-4dfd-afc3-761a390bc583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:58.840094 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8fde63-23ca-4dfd-afc3-761a390bc583 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:58.840431 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8fde63-23ca-4dfd-afc3-761a390bc583 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:58.840631 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8fde63-23ca-4dfd-afc3-761a390bc583 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:58.841249 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8fde63-23ca-4dfd-afc3-761a390bc583 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:58.841551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8fde63-23ca-4dfd-afc3-761a390bc583 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:58.842942 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8fde63-23ca-4dfd-afc3-761a390bc583 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:58.854701 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 413/845] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Sep 10 23:42:58 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:58.854701 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde485d-43b1-4cf3-9088-3e83b7c22205 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:58.854701 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde485d-43b1-4cf3-9088-3e83b7c22205 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:58.854701 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde485d-43b1-4cf3-9088-3e83b7c22205 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:58.854701 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 414/846] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:42:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:42:58.865303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c97d1d-01e7-4e29-90dd-195e1a982da1 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:58.865844 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c97d1d-01e7-4e29-90dd-195e1a982da1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:58.866245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c97d1d-01e7-4e29-90dd-195e1a982da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:58.875567 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b6517-b83c-47c6-9b59-71371612f755 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:58.877190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b6517-b83c-47c6-9b59-71371612f755 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:58.892391 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-03c97d1d-01e7-4e29-90dd-195e1a982da1 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:42:58.893887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-03c97d1d-01e7-4e29-90dd-195e1a982da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:42:58.951185 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:58.994185 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 415/847] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:42:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 23:42:59.001004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 433/848] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:42:58 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 23:42:59.010026 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c97d1d-01e7-4e29-90dd-195e1a982da1 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:59.084143 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:59.118552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c97d1d-01e7-4e29-90dd-195e1a982da1 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:59.154294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43269f34-ae8c-4934-b8bc-a6f351f7f0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:59.154958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43269f34-ae8c-4934-b8bc-a6f351f7f0ce None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:59.155271 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43269f34-ae8c-4934-b8bc-a6f351f7f0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:59.155539 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43269f34-ae8c-4934-b8bc-a6f351f7f0ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:59.165990 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43269f34-ae8c-4934-b8bc-a6f351f7f0ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:59.167676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43269f34-ae8c-4934-b8bc-a6f351f7f0ce None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:59.183566 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-43269f34-ae8c-4934-b8bc-a6f351f7f0ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:59.183566 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:59.183566 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:59.183566 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:59.183566 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:59.183566 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:59.183566 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:59.183566 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:59.183566 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:59.183566 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:59.183566 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:59.183566 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:59.183566 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:42:59.183566 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:42:59.183566 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:42:59.183566 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:42:59.184545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:59.184545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:59.184545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:59.184545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:59.184545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:59.184545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:59.184545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:59.184545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:59.184545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:59.184545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:59.184545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:59.184545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:59.184545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:59.184545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:59.184545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:59.184545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:59.184545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:42:59.185429 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:42:59.185429 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:42:59.185429 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:59.185429 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 416/849] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:42:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:42:59.193901 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b6517-b83c-47c6-9b59-71371612f755 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:59.285747 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43269f34-ae8c-4934-b8bc-a6f351f7f0ce None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:59.296106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b6517-b83c-47c6-9b59-71371612f755 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:59.337182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db3adb-87ea-4d48-ae3e-7fc856bbb727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:59.337876 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db3adb-87ea-4d48-ae3e-7fc856bbb727 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:59.338221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db3adb-87ea-4d48-ae3e-7fc856bbb727 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:59.339406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db3adb-87ea-4d48-ae3e-7fc856bbb727 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:59.340134 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db3adb-87ea-4d48-ae3e-7fc856bbb727 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:59.340503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db3adb-87ea-4d48-ae3e-7fc856bbb727 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:59.341931 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db3adb-87ea-4d48-ae3e-7fc856bbb727 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:59.357248 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 434/850] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:59.362984 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db3adb-87ea-4d48-ae3e-7fc856bbb727 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:59.396119 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:59.410485 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43269f34-ae8c-4934-b8bc-a6f351f7f0ce None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:59.433088 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:59.452307 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca844a-d1fc-4a95-9076-84f6907613c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:59.453757 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca844a-d1fc-4a95-9076-84f6907613c0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:59.454639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca844a-d1fc-4a95-9076-84f6907613c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:59.455269 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca844a-d1fc-4a95-9076-84f6907613c0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:59.456132 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca844a-d1fc-4a95-9076-84f6907613c0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:59.458440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca844a-d1fc-4a95-9076-84f6907613c0 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:59.471277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-cbca844a-d1fc-4a95-9076-84f6907613c0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 23:42:59.471277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:59.471277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 23:42:59.471277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 23:42:59.471277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 23:42:59.471277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 23:42:59.471277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 23:42:59.471277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:42:59.471277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 23:42:59.471277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:59.471277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:42:59.471277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:59.471277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:59.471277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:59.471277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:59.471277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:59.472217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:59.472217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:59.472217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:59.472217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:59.472217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:59.472217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:59.472217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:59.472217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 23:42:59.472217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 23:42:59.472217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 23:42:59.472217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 23:42:59.472217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:59.472217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:59.472217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:59.472217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:59.472217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:59.472217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:59.473109 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:59.473109 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:59.473109 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:59.473109 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:59.473109 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:59.473109 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:59.473109 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:59.473109 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:59.473109 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:59.473109 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:59.473109 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 23:42:59.473109 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:42:59.473109 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 23:42:59.473109 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:42:59.481178 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 417/851] 10.222.0.17 () {56 vars in 1132 bytes} [Sat Sep 10 23:42:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:42:59.483531 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca844a-d1fc-4a95-9076-84f6907613c0 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:59.486215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db3adb-87ea-4d48-ae3e-7fc856bbb727 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:59.524643 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cafbd1-4407-4e2e-a8f9-92056411b32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:59.525393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cafbd1-4407-4e2e-a8f9-92056411b32a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:59.525581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cafbd1-4407-4e2e-a8f9-92056411b32a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:59.525852 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cafbd1-4407-4e2e-a8f9-92056411b32a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:59.533058 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cafbd1-4407-4e2e-a8f9-92056411b32a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:59.534501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cafbd1-4407-4e2e-a8f9-92056411b32a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:59.548555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-b6cafbd1-4407-4e2e-a8f9-92056411b32a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:59.548555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:42:59.548555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:42:59.548555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:42:59.548555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:42:59.548555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:42:59.548555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:42:59.548555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:42:59.548555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:42:59.548555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:42:59.548555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:42:59.548555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:42:59.548555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:42:59.548555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:42:59.548555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:42:59.549449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:42:59.549449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:42:59.549449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:42:59.549449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:42:59.549449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:42:59.549449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:42:59.549449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:42:59.549449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:42:59.549449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:42:59.549449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:42:59.549449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:42:59.549449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:42:59.549449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:42:59.549449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:42:59.549449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:42:59.549449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:42:59.549449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:42:59.550328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:42:59.550328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:42:59.550328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:42:59.550328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:42:59.550328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:42:59.550328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:42:59.550328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:42:59.550328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:42:59.554496 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 435/852] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:42:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:42:59.555540 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cafbd1-4407-4e2e-a8f9-92056411b32a None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:59.596106 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca844a-d1fc-4a95-9076-84f6907613c0 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:59.632238 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b2eb7f-9a41-43fb-9c49-59a9646d024f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:59.632986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b2eb7f-9a41-43fb-9c49-59a9646d024f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:59.633355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b2eb7f-9a41-43fb-9c49-59a9646d024f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:59.633720 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b2eb7f-9a41-43fb-9c49-59a9646d024f None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:59.634485 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b2eb7f-9a41-43fb-9c49-59a9646d024f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:59.634928 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b2eb7f-9a41-43fb-9c49-59a9646d024f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:59.637333 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b2eb7f-9a41-43fb-9c49-59a9646d024f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:59.646900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 418/853] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Sep 10 23:42:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:59.648673 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b2eb7f-9a41-43fb-9c49-59a9646d024f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:59.662441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:59.677617 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cafbd1-4407-4e2e-a8f9-92056411b32a None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:59.701745 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:42:59.718728 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5edba27-bac4-46a1-802a-d75013bd750e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:59.719609 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5edba27-bac4-46a1-802a-d75013bd750e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:59.720072 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5edba27-bac4-46a1-802a-d75013bd750e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:59.720480 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5edba27-bac4-46a1-802a-d75013bd750e None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:59.721343 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5edba27-bac4-46a1-802a-d75013bd750e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:59.721799 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5edba27-bac4-46a1-802a-d75013bd750e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:59.723616 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5edba27-bac4-46a1-802a-d75013bd750e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:59.740899 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 436/854] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:42:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:42:59.752362 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5edba27-bac4-46a1-802a-d75013bd750e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:42:59.764500 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b2eb7f-9a41-43fb-9c49-59a9646d024f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:59.812072 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416ed74-44b7-49c8-8f38-a4c29246d9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:59.812609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416ed74-44b7-49c8-8f38-a4c29246d9dd None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:59.812891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416ed74-44b7-49c8-8f38-a4c29246d9dd None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:59.813438 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416ed74-44b7-49c8-8f38-a4c29246d9dd None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:59.814931 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6416ed74-44b7-49c8-8f38-a4c29246d9dd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:59.815255 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6416ed74-44b7-49c8-8f38-a4c29246d9dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:59.819033 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6416ed74-44b7-49c8-8f38-a4c29246d9dd None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:59.826538 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 419/855] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Sep 10 23:42:59 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:42:59.898223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5edba27-bac4-46a1-802a-d75013bd750e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:42:59.938742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27feb7-962f-4dbc-974d-6fe1879ba18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:42:59.939360 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27feb7-962f-4dbc-974d-6fe1879ba18b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:42:59.939639 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27feb7-962f-4dbc-974d-6fe1879ba18b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:42:59.939908 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27feb7-962f-4dbc-974d-6fe1879ba18b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:42:59.940646 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27feb7-962f-4dbc-974d-6fe1879ba18b None admin] RBAC: Adding query filter params (role.id=9dce48a2b53c482d931dcd064b07744e, scope.project.id=947277c4458645899287e6818947c076, user.id=564e65585e9543e286b6041af870cf41) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:42:59.940955 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27feb7-962f-4dbc-974d-6fe1879ba18b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:42:59.942488 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27feb7-962f-4dbc-974d-6fe1879ba18b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:42:59.949874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 437/856] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:42:59 2022] GET /identity/v3/role_assignments?user.id=564e65585e9543e286b6041af870cf41&scope.project.id=947277c4458645899287e6818947c076&role.id=9dce48a2b53c482d931dcd064b07744e => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:00.145730 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c076c-e2d6-43ba-9f7e-ae0b4da03443 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:00.146027 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c076c-e2d6-43ba-9f7e-ae0b4da03443 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:00.146212 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c076c-e2d6-43ba-9f7e-ae0b4da03443 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:00.147787 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 420/857] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:00.153420 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca1343-0183-4bbb-8046-3d805dd82d5a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:00.153694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca1343-0183-4bbb-8046-3d805dd82d5a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:00.153967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca1343-0183-4bbb-8046-3d805dd82d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:00.173416 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-19ca1343-0183-4bbb-8046-3d805dd82d5a None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:00.174608 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-19ca1343-0183-4bbb-8046-3d805dd82d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:00.183843 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:00.245552 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 438/858] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:43:00.515867 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c076c-e2d6-43ba-9f7e-ae0b4da03443 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:00.623338 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c076c-e2d6-43ba-9f7e-ae0b4da03443 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:00.637933 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:00.659547 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531cb0e-7e57-4447-abab-72beb2844f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:00.660456 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531cb0e-7e57-4447-abab-72beb2844f95 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:00.660883 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531cb0e-7e57-4447-abab-72beb2844f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:00.661315 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531cb0e-7e57-4447-abab-72beb2844f95 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:00.672620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531cb0e-7e57-4447-abab-72beb2844f95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:43:00.677645 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-4531cb0e-7e57-4447-abab-72beb2844f95 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:43:00.677645 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:00.677645 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:00.677645 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:00.677645 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:00.677645 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:00.677645 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:00.677645 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:00.677645 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:00.677645 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:00.677645 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:00.677645 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:00.677645 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:43:00.677645 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:43:00.677645 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:43:00.677645 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:43:00.678807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:43:00.678807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:43:00.678807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:43:00.678807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:43:00.678807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:00.678807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:00.678807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:00.678807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:00.678807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:00.678807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:00.678807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:00.678807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:00.678807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:00.678807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:00.678807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:00.678807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:00.679858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:00.679858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:00.679858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:00.679858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:00.679858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:43:00.679858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:43:00.679858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:43:00.679858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:43:00.679858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:43:00.679858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:43:00.679858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:43:00.679858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:00.682544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 421/859] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Sep 10 23:43:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:00.694724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ca1343-0183-4bbb-8046-3d805dd82d5a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:00.794587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ca1343-0183-4bbb-8046-3d805dd82d5a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:00.834398 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2f774-cb93-49c3-a5d0-513f5563a2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:00.835045 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2f774-cb93-49c3-a5d0-513f5563a2f8 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:00.835318 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2f774-cb93-49c3-a5d0-513f5563a2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:00.835586 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2f774-cb93-49c3-a5d0-513f5563a2f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:00.836250 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b2f774-cb93-49c3-a5d0-513f5563a2f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:00.836548 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b2f774-cb93-49c3-a5d0-513f5563a2f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:00.838051 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b2f774-cb93-49c3-a5d0-513f5563a2f8 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:00.847177 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 439/860] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Sep 10 23:43:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:00.852866 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531cb0e-7e57-4447-abab-72beb2844f95 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:00.968661 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531cb0e-7e57-4447-abab-72beb2844f95 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:01.011564 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee776ab-ed9d-4ba3-a42e-c24625af6cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:01.012709 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee776ab-ed9d-4ba3-a42e-c24625af6cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:01.013190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee776ab-ed9d-4ba3-a42e-c24625af6cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:01.013673 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee776ab-ed9d-4ba3-a42e-c24625af6cc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:01.025764 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee776ab-ed9d-4ba3-a42e-c24625af6cc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:01.027796 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee776ab-ed9d-4ba3-a42e-c24625af6cc1 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:01.043886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-eee776ab-ed9d-4ba3-a42e-c24625af6cc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:01.043886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:01.043886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:01.043886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:01.043886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:01.043886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:01.043886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:01.043886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:01.043886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:01.043886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:01.043886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:01.043886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:01.043886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:43:01.043886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:43:01.043886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:43:01.043886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:43:01.044878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:01.044878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:01.044878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:01.044878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:01.044878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:01.044878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:01.044878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:01.044878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:01.044878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:01.044878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:01.044878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:01.044878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:01.044878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:01.044878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:01.044878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:01.044878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:01.044878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:43:01.045792 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:43:01.045792 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:01.045792 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:01.048053 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 422/861] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:43:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:01.053433 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2f774-cb93-49c3-a5d0-513f5563a2f8 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:01.071025 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:01.103908 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5149da-258e-4f1d-b635-0768e4b36508 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:01.104234 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5149da-258e-4f1d-b635-0768e4b36508 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:01.104965 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5149da-258e-4f1d-b635-0768e4b36508 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:01.106571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 423/862] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:43:01.111630 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562bfd0-c0f5-475f-aad2-b0ee6fd5e1d5 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:01.111974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562bfd0-c0f5-475f-aad2-b0ee6fd5e1d5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:01.112313 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562bfd0-c0f5-475f-aad2-b0ee6fd5e1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:01.133405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a562bfd0-c0f5-475f-aad2-b0ee6fd5e1d5 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:01.134715 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a562bfd0-c0f5-475f-aad2-b0ee6fd5e1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:01.160698 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2f774-cb93-49c3-a5d0-513f5563a2f8 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:01.171025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:01.196104 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7ca6e-912e-42cb-b853-e87e7f50b820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:01.196709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7ca6e-912e-42cb-b853-e87e7f50b820 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:01.197058 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7ca6e-912e-42cb-b853-e87e7f50b820 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:01.197329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7ca6e-912e-42cb-b853-e87e7f50b820 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:01.197973 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a7ca6e-912e-42cb-b853-e87e7f50b820 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:01.198312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a7ca6e-912e-42cb-b853-e87e7f50b820 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:01.199733 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a7ca6e-912e-42cb-b853-e87e7f50b820 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:01.211985 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 440/863] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:01.218136 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7ca6e-912e-42cb-b853-e87e7f50b820 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:01.221342 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 424/864] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:43:01.311598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:01.326986 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7ca6e-912e-42cb-b853-e87e7f50b820 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:01.362634 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fba0f2-86fa-4a30-83d9-e0ac36b0825e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:01.363286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fba0f2-86fa-4a30-83d9-e0ac36b0825e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:01.363560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fba0f2-86fa-4a30-83d9-e0ac36b0825e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:01.363825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fba0f2-86fa-4a30-83d9-e0ac36b0825e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:01.371709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fba0f2-86fa-4a30-83d9-e0ac36b0825e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:01.373134 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fba0f2-86fa-4a30-83d9-e0ac36b0825e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:01.384766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-08fba0f2-86fa-4a30-83d9-e0ac36b0825e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 23:43:01.384766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:01.384766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:01.384766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:01.384766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:01.384766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:01.384766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:01.384766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:01.384766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:01.384766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:01.384766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:01.384766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:01.384766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:43:01.384766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:43:01.384766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:43:01.384766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:43:01.385737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:01.385737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:01.385737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:43:01.385737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:43:01.385737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:43:01.385737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:43:01.385737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:01.385737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:01.385737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:01.385737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:01.385737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:01.385737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:01.385737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:01.385737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:01.385737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:01.385737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:01.385737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:01.385737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:01.386778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:01.386778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:01.386778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:43:01.386778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:43:01.386778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:43:01.386778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:43:01.386778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:43:01.386778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:43:01.386778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 23:43:01.386778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:01.386778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 441/865] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Sep 10 23:43:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:43:01.391886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a562bfd0-c0f5-475f-aad2-b0ee6fd5e1d5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:01.494791 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a562bfd0-c0f5-475f-aad2-b0ee6fd5e1d5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:01.513132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fba0f2-86fa-4a30-83d9-e0ac36b0825e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:01.536795 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f646704-44a9-454e-834f-7d0d187ed47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:01.537432 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f646704-44a9-454e-834f-7d0d187ed47f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:01.537659 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f646704-44a9-454e-834f-7d0d187ed47f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:01.537927 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f646704-44a9-454e-834f-7d0d187ed47f None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:01.538647 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f646704-44a9-454e-834f-7d0d187ed47f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:01.538971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f646704-44a9-454e-834f-7d0d187ed47f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:01.540388 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f646704-44a9-454e-834f-7d0d187ed47f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:01.559179 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 425/866] 10.222.0.17 () {56 vars in 1149 bytes} [Sat Sep 10 23:43:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:01.567611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f646704-44a9-454e-834f-7d0d187ed47f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:01.587212 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:01.652144 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fba0f2-86fa-4a30-83d9-e0ac36b0825e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:01.686163 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:01.700788 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf1ab7-8fbf-4d93-9319-4280f3b060d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:01.701727 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f646704-44a9-454e-834f-7d0d187ed47f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:01.702229 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf1ab7-8fbf-4d93-9319-4280f3b060d5 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:01.702695 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf1ab7-8fbf-4d93-9319-4280f3b060d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:01.703111 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf1ab7-8fbf-4d93-9319-4280f3b060d5 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:01.704183 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf1ab7-8fbf-4d93-9319-4280f3b060d5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:01.706695 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf1ab7-8fbf-4d93-9319-4280f3b060d5 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:01.740903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86d10a-8db1-4cfd-af1f-5a139adbd6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:01.741614 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86d10a-8db1-4cfd-af1f-5a139adbd6f1 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:01.741894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86d10a-8db1-4cfd-af1f-5a139adbd6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:01.742198 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86d10a-8db1-4cfd-af1f-5a139adbd6f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:01.778751 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 442/867] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Sep 10 23:43:01 2022] POST /identity/v3/services => generated 251 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 23:43:01.838041 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86d10a-8db1-4cfd-af1f-5a139adbd6f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:01.840747 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86d10a-8db1-4cfd-af1f-5a139adbd6f1 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:01.858319 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:01.956668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 426/868] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:43:01 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 23:43:01.962654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf1ab7-8fbf-4d93-9319-4280f3b060d5 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:02.096353 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf1ab7-8fbf-4d93-9319-4280f3b060d5 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:02.111987 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7049f5-debc-40ff-b6c1-ec8900d55faf None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:02.112263 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7049f5-debc-40ff-b6c1-ec8900d55faf None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:02.112849 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7049f5-debc-40ff-b6c1-ec8900d55faf None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:02.115257 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 427/869] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:43:02.119424 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74aea7-8fad-448d-932e-786a589c7193 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:02.119424 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74aea7-8fad-448d-932e-786a589c7193 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:02.119652 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74aea7-8fad-448d-932e-786a589c7193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:02.138106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6106dd-a6af-42b0-aaca-c7068de13269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:02.138769 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6106dd-a6af-42b0-aaca-c7068de13269 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:02.139045 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6106dd-a6af-42b0-aaca-c7068de13269 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:02.139307 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6106dd-a6af-42b0-aaca-c7068de13269 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:02.140295 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-bb74aea7-8fad-448d-932e-786a589c7193 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:02.141775 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-bb74aea7-8fad-448d-932e-786a589c7193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:02.149685 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6106dd-a6af-42b0-aaca-c7068de13269 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:02.151157 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6106dd-a6af-42b0-aaca-c7068de13269 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:02.166553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-ce6106dd-a6af-42b0-aaca-c7068de13269 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:02.166553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:02.166553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:02.166553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:02.166553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:02.166553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:02.166553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:02.166553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:02.166553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:02.166553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:02.166553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:02.166553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:02.166553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:43:02.166553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:43:02.166553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:43:02.166553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:43:02.167980 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:02.167980 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:02.167980 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:02.167980 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:02.167980 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:02.167980 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:02.167980 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:02.167980 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:02.167980 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:02.167980 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:02.167980 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:02.167980 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:02.167980 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:02.167980 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:02.167980 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:02.167980 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:02.167980 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:43:02.168883 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:43:02.168883 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:02.168883 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:02.168883 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 443/870] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:43:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:02.175325 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6106dd-a6af-42b0-aaca-c7068de13269 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:02.218522 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:02.236366 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 428/871] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:02 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 23:43:02.291947 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf964f79-7de1-4f64-8ad5-27e785e65a84 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:02.292284 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf964f79-7de1-4f64-8ad5-27e785e65a84 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:02.292581 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf964f79-7de1-4f64-8ad5-27e785e65a84 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:02.294870 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 429/872] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:43:02.300222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9ea63-2a2e-428a-b609-6fd2928d771b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:02.300507 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9ea63-2a2e-428a-b609-6fd2928d771b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:02.300783 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9ea63-2a2e-428a-b609-6fd2928d771b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:02.313491 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6106dd-a6af-42b0-aaca-c7068de13269 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:02.322708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-89d9ea63-2a2e-428a-b609-6fd2928d771b None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:02.326122 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-89d9ea63-2a2e-428a-b609-6fd2928d771b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:02.349769 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3e2f4-f375-4b2b-9391-a3597af9d920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:02.350723 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3e2f4-f375-4b2b-9391-a3597af9d920 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:02.351196 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3e2f4-f375-4b2b-9391-a3597af9d920 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:02.351836 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3e2f4-f375-4b2b-9391-a3597af9d920 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:02.352712 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a3e2f4-f375-4b2b-9391-a3597af9d920 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:02.353155 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a3e2f4-f375-4b2b-9391-a3597af9d920 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:02.355652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a3e2f4-f375-4b2b-9391-a3597af9d920 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:02.369037 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 444/873] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:02.371823 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:02.375740 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3e2f4-f375-4b2b-9391-a3597af9d920 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:02.408777 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 430/874] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:02 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 23:43:02.480067 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:02.489591 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3e2f4-f375-4b2b-9391-a3597af9d920 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:02.522483 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d9ea63-2a2e-428a-b609-6fd2928d771b None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:02.526574 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf1509-2606-4ac1-9847-0159883e1066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:02.527259 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf1509-2606-4ac1-9847-0159883e1066 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:02.527538 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf1509-2606-4ac1-9847-0159883e1066 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:02.527813 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf1509-2606-4ac1-9847-0159883e1066 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:02.535386 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf1509-2606-4ac1-9847-0159883e1066 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:02.536831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf1509-2606-4ac1-9847-0159883e1066 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:02.551227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-4ecf1509-2606-4ac1-9847-0159883e1066 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:43:02.551227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:02.551227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:02.551227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:02.551227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:02.551227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:02.551227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:02.551227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:02.551227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:02.551227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:02.551227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:02.551227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:02.551227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:43:02.551227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:43:02.551227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:43:02.552100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:43:02.552100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:02.552100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:02.552100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:02.552100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:02.552100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:02.552100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:02.552100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:02.552100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:02.552100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:02.552100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:02.552100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:02.552100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:02.552100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:02.552100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:02.552100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:02.552100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:02.553652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:43:02.553652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:43:02.553652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:43:02.553652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:43:02.553652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:43:02.553652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:43:02.553652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:43:02.553652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:02.553652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 445/875] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:43:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:02.558879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf1509-2606-4ac1-9847-0159883e1066 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:02.649203 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d9ea63-2a2e-428a-b609-6fd2928d771b None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:02.667858 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf1509-2606-4ac1-9847-0159883e1066 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:02.695834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292bf60-26b7-4ebf-8194-f24a4205d59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:02.696440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292bf60-26b7-4ebf-8194-f24a4205d59f None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:02.696719 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292bf60-26b7-4ebf-8194-f24a4205d59f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:02.696985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292bf60-26b7-4ebf-8194-f24a4205d59f None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:02.697639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292bf60-26b7-4ebf-8194-f24a4205d59f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:02.699076 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292bf60-26b7-4ebf-8194-f24a4205d59f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:02.705841 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6664edf7-d15a-4777-b370-7fc2497d2e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:02.706436 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664edf7-d15a-4777-b370-7fc2497d2e55 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:02.706732 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664edf7-d15a-4777-b370-7fc2497d2e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:02.706998 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664edf7-d15a-4777-b370-7fc2497d2e55 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:02.707725 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6664edf7-d15a-4777-b370-7fc2497d2e55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:02.708023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6664edf7-d15a-4777-b370-7fc2497d2e55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:02.709420 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6664edf7-d15a-4777-b370-7fc2497d2e55 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:02.722488 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 446/876] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:43:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:02.723398 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6664edf7-d15a-4777-b370-7fc2497d2e55 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:02.742756 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:02.764084 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 431/877] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Sep 10 23:43:02 2022] POST /identity/v3/services => generated 269 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 23:43:02.764924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292bf60-26b7-4ebf-8194-f24a4205d59f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:02.815200 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:02.853519 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6664edf7-d15a-4777-b370-7fc2497d2e55 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:02.913017 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292bf60-26b7-4ebf-8194-f24a4205d59f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:02.915284 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba946afd-7af1-4a72-a71e-6ea0c44878e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:02.917158 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba946afd-7af1-4a72-a71e-6ea0c44878e1 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:02.917599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba946afd-7af1-4a72-a71e-6ea0c44878e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:02.918011 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba946afd-7af1-4a72-a71e-6ea0c44878e1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:02.918766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba946afd-7af1-4a72-a71e-6ea0c44878e1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:02.920358 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba946afd-7af1-4a72-a71e-6ea0c44878e1 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:02.933824 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-ba946afd-7af1-4a72-a71e-6ea0c44878e1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 23:43:02.933824 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:02.933824 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 23:43:02.933824 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 23:43:02.933824 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 23:43:02.933824 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 23:43:02.933824 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 23:43:02.933824 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:02.933824 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 23:43:02.933824 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:02.933824 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:43:02.933824 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:02.933824 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:02.933824 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:02.933824 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:02.933824 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:02.933824 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:02.935079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:02.935079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:02.935079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:02.935079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:02.935079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:02.935079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:02.935079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 23:43:02.935079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 23:43:02.935079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 23:43:02.935079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 23:43:02.935079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:02.935079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:02.935079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:02.935079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:02.935079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:02.935079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:02.936101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:02.936101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:02.936101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:02.936101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:02.936101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:02.936101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:02.936101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:02.936101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:02.936101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:02.936101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:02.936101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 23:43:02.936101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:02.936101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 23:43:02.936101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:02.944245 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 447/878] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Sep 10 23:43:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:02.944871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba946afd-7af1-4a72-a71e-6ea0c44878e1 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:02.963408 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585956f2-fcf5-40ab-9694-54ad15464673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:02.964217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585956f2-fcf5-40ab-9694-54ad15464673 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:02.964509 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585956f2-fcf5-40ab-9694-54ad15464673 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:02.964785 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585956f2-fcf5-40ab-9694-54ad15464673 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:02.965511 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585956f2-fcf5-40ab-9694-54ad15464673 None admin] RBAC: Adding query filter params (role.id=8ff1727b65104e5fb97333d477196d46, scope.project.id=947277c4458645899287e6818947c076, user.id=b0a879085a194b0687e1cdcbafd48033) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:02.965839 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585956f2-fcf5-40ab-9694-54ad15464673 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:02.967454 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585956f2-fcf5-40ab-9694-54ad15464673 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:02.975062 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 432/879] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:43:02 2022] GET /identity/v3/role_assignments?user.id=b0a879085a194b0687e1cdcbafd48033&scope.project.id=947277c4458645899287e6818947c076&role.id=8ff1727b65104e5fb97333d477196d46 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:03.040409 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:03.075005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba946afd-7af1-4a72-a71e-6ea0c44878e1 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:03.113755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b2dc5-6c43-419e-8508-dd43bf1b70dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:03.114372 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b2dc5-6c43-419e-8508-dd43bf1b70dd None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:03.114644 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b2dc5-6c43-419e-8508-dd43bf1b70dd None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:03.114948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b2dc5-6c43-419e-8508-dd43bf1b70dd None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:03.115582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b2dc5-6c43-419e-8508-dd43bf1b70dd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:03.115881 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b2dc5-6c43-419e-8508-dd43bf1b70dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:03.117261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b2dc5-6c43-419e-8508-dd43bf1b70dd None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:03.123236 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 448/880] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Sep 10 23:43:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:03.128710 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585956f2-fcf5-40ab-9694-54ad15464673 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:03.231261 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:03.243347 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585956f2-fcf5-40ab-9694-54ad15464673 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:03.284428 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864deb8-5f77-4344-b409-a6deb7c5ae16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:03.285030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864deb8-5f77-4344-b409-a6deb7c5ae16 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:03.285354 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864deb8-5f77-4344-b409-a6deb7c5ae16 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:03.285631 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864deb8-5f77-4344-b409-a6deb7c5ae16 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:03.286263 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7864deb8-5f77-4344-b409-a6deb7c5ae16 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:03.286565 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7864deb8-5f77-4344-b409-a6deb7c5ae16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:03.288023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7864deb8-5f77-4344-b409-a6deb7c5ae16 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:03.293551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 433/881] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Sep 10 23:43:03 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:04.177234 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567fb1c-bcac-44a4-af1d-fc2b880301aa None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:04.177897 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567fb1c-bcac-44a4-af1d-fc2b880301aa None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:04.178116 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567fb1c-bcac-44a4-af1d-fc2b880301aa None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:04.180046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 449/882] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:04.187216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e4877-a84d-49e8-8bc0-0374cce59cb3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:04.187754 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e4877-a84d-49e8-8bc0-0374cce59cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:04.187754 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e4877-a84d-49e8-8bc0-0374cce59cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:04.208524 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-571e4877-a84d-49e8-8bc0-0374cce59cb3 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:04.209707 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-571e4877-a84d-49e8-8bc0-0374cce59cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:04.282816 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 434/883] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:04 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 23:43:04.551299 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b567fb1c-bcac-44a4-af1d-fc2b880301aa None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:04.659109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b567fb1c-bcac-44a4-af1d-fc2b880301aa None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:04.694363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d907b5-f3f8-42aa-9e83-5470ce44b783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:04.695017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d907b5-f3f8-42aa-9e83-5470ce44b783 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:04.695294 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d907b5-f3f8-42aa-9e83-5470ce44b783 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:04.695568 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d907b5-f3f8-42aa-9e83-5470ce44b783 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:04.696312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d907b5-f3f8-42aa-9e83-5470ce44b783 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:04.697791 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d907b5-f3f8-42aa-9e83-5470ce44b783 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:04.701161 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:04.707978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-c4d907b5-f3f8-42aa-9e83-5470ce44b783 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 23:43:04.707978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:04.707978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 23:43:04.707978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 23:43:04.707978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 23:43:04.707978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 23:43:04.707978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 23:43:04.707978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:04.707978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 23:43:04.707978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:04.707978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:43:04.707978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:04.707978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:04.707978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:04.707978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:04.707978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:04.707978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:04.709023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:04.709023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:04.709023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:04.709023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:04.709023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:04.709023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:04.709023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 23:43:04.709023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 23:43:04.709023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 23:43:04.709023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 23:43:04.709023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:04.709023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:04.709023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:04.709023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:04.709023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:04.709023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:04.709919 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:04.709919 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:04.709919 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:04.709919 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:04.709919 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:04.709919 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:04.709919 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:04.709919 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:04.709919 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:04.709919 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:04.709919 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 23:43:04.709919 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:04.709919 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 23:43:04.709919 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:04.709919 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 450/884] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Sep 10 23:43:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:04.715813 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571e4877-a84d-49e8-8bc0-0374cce59cb3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:04.807824 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:04.824593 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571e4877-a84d-49e8-8bc0-0374cce59cb3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:04.866565 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c14cb4-1c49-4989-8be3-d926375e2a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:04.867577 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c14cb4-1c49-4989-8be3-d926375e2a3b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:04.868054 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c14cb4-1c49-4989-8be3-d926375e2a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:04.868532 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c14cb4-1c49-4989-8be3-d926375e2a3b None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:04.869392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c14cb4-1c49-4989-8be3-d926375e2a3b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:04.869905 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c14cb4-1c49-4989-8be3-d926375e2a3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:04.871655 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c14cb4-1c49-4989-8be3-d926375e2a3b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:04.877046 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 435/885] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Sep 10 23:43:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:04.882689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d907b5-f3f8-42aa-9e83-5470ce44b783 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:04.987639 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d907b5-f3f8-42aa-9e83-5470ce44b783 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:05.023689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1724527-bf2c-43f6-b71f-fed80116c736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:05.024296 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1724527-bf2c-43f6-b71f-fed80116c736 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:05.024576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1724527-bf2c-43f6-b71f-fed80116c736 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:05.024856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1724527-bf2c-43f6-b71f-fed80116c736 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:05.025490 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1724527-bf2c-43f6-b71f-fed80116c736 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:05.025794 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1724527-bf2c-43f6-b71f-fed80116c736 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:05.027786 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1724527-bf2c-43f6-b71f-fed80116c736 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:05.057593 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 451/886] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Sep 10 23:43:04 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:05.063637 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c14cb4-1c49-4989-8be3-d926375e2a3b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:05.174958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c14cb4-1c49-4989-8be3-d926375e2a3b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:05.190179 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67805ad-58a4-4824-bb52-8a3b36480960 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:05.190718 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67805ad-58a4-4824-bb52-8a3b36480960 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:05.190829 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67805ad-58a4-4824-bb52-8a3b36480960 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:05.192188 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 452/887] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:05.197641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe08b1c-e231-458a-b2b1-f83a6edeb646 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:05.198333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe08b1c-e231-458a-b2b1-f83a6edeb646 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:05.199137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe08b1c-e231-458a-b2b1-f83a6edeb646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:05.219231 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-6fe08b1c-e231-458a-b2b1-f83a6edeb646 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:05.220451 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455702a-e1a5-4c7b-9161-608dc001f7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:05.220925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6fe08b1c-e231-458a-b2b1-f83a6edeb646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:05.221735 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455702a-e1a5-4c7b-9161-608dc001f7aa None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:05.222213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455702a-e1a5-4c7b-9161-608dc001f7aa None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:05.222797 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455702a-e1a5-4c7b-9161-608dc001f7aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:05.223840 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455702a-e1a5-4c7b-9161-608dc001f7aa None admin] RBAC: Adding query filter params (interface=public, service_id=ca7d44bb654e465786b2fb37a44a98ba, region_id=RegionOne) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:05.224145 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455702a-e1a5-4c7b-9161-608dc001f7aa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:05.225505 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455702a-e1a5-4c7b-9161-608dc001f7aa None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:05.231582 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 436/888] 10.222.0.17 () {56 vars in 1251 bytes} [Sat Sep 10 23:43:05 2022] GET /identity/v3/endpoints?service_id=ca7d44bb654e465786b2fb37a44a98ba&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:05.237011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455702a-e1a5-4c7b-9161-608dc001f7aa None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:05.254009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:05.285048 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:05.309863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 453/889] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:43:05.348382 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455702a-e1a5-4c7b-9161-608dc001f7aa None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:05.394536 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8a719-ef1b-4e44-a3a2-433a8b6168a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:05.395248 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8a719-ef1b-4e44-a3a2-433a8b6168a7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:05.395527 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8a719-ef1b-4e44-a3a2-433a8b6168a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:05.395796 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8a719-ef1b-4e44-a3a2-433a8b6168a7 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:05.396462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8a719-ef1b-4e44-a3a2-433a8b6168a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:05.397896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8a719-ef1b-4e44-a3a2-433a8b6168a7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:05.404234 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 437/890] 10.222.0.17 () {56 vars in 1087 bytes} [Sat Sep 10 23:43:05 2022] GET /identity/v3/services => generated 1268 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 23:43:05.478636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc13a3-b77e-4da6-86a2-148d823ec058 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:05.478636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc13a3-b77e-4da6-86a2-148d823ec058 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:05.478636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc13a3-b77e-4da6-86a2-148d823ec058 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:05.480340 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5f5d6-738c-4631-b52a-f0cbbe25b0e8 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:05.480340 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5f5d6-738c-4631-b52a-f0cbbe25b0e8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:05.480340 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5f5d6-738c-4631-b52a-f0cbbe25b0e8 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:05.481199 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 454/891] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:05.481910 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 438/892] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:43:05.485942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7855fd-ab90-486d-8f30-7bde4dc4a9be None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:05.486213 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7855fd-ab90-486d-8f30-7bde4dc4a9be None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:05.486483 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7855fd-ab90-486d-8f30-7bde4dc4a9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:05.489390 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4f9fc-b9e0-4377-8b78-e0e3d07561e5 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:05.489645 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4f9fc-b9e0-4377-8b78-e0e3d07561e5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:05.489971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4f9fc-b9e0-4377-8b78-e0e3d07561e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:05.512409 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-3c7855fd-ab90-486d-8f30-7bde4dc4a9be None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:05.513170 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a1f4f9fc-b9e0-4377-8b78-e0e3d07561e5 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:05.514369 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a1f4f9fc-b9e0-4377-8b78-e0e3d07561e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:05.514631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3c7855fd-ab90-486d-8f30-7bde4dc4a9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:05.611976 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 455/893] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 23:43:05.618780 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 439/894] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 23:43:05.635268 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f4f9fc-b9e0-4377-8b78-e0e3d07561e5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:05.649916 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:05.766936 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f4f9fc-b9e0-4377-8b78-e0e3d07561e5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:05.811048 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10c16e-0261-4777-8515-48f517e614e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:05.811694 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10c16e-0261-4777-8515-48f517e614e7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:05.812026 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10c16e-0261-4777-8515-48f517e614e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:05.812304 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10c16e-0261-4777-8515-48f517e614e7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:05.812895 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10c16e-0261-4777-8515-48f517e614e7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:05.814270 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10c16e-0261-4777-8515-48f517e614e7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:05.826127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-5e10c16e-0261-4777-8515-48f517e614e7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 23:43:05.826127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:05.826127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 23:43:05.826127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 23:43:05.826127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 23:43:05.826127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 23:43:05.826127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 23:43:05.826127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:05.826127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 23:43:05.826127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:05.826127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:43:05.826127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:05.826127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:05.826127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:05.826127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:05.826127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:05.828077 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:05.828077 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:05.828077 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:05.828077 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:05.828077 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:05.828077 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:05.828077 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:05.828077 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 23:43:05.828077 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 23:43:05.828077 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 23:43:05.828077 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 23:43:05.828077 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:05.828077 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:05.828077 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:05.828077 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:05.828077 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:05.828077 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:05.832408 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:05.832408 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:05.832408 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:05.832408 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:05.832408 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:05.832408 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:05.832408 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:05.832408 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:05.832408 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:05.832408 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:05.832408 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 23:43:05.832408 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:05.832408 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 23:43:05.832408 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:05.832408 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 440/895] 10.222.0.17 () {56 vars in 1132 bytes} [Sat Sep 10 23:43:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:05.837177 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7855fd-ab90-486d-8f30-7bde4dc4a9be None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:05.909327 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10c16e-0261-4777-8515-48f517e614e7 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:05.968495 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7855fd-ab90-486d-8f30-7bde4dc4a9be None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:05.997168 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:06.004570 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:06.019835 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3cad93-53d8-49df-8a2b-5b8fce224611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:06.020786 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3cad93-53d8-49df-8a2b-5b8fce224611 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:06.021353 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3cad93-53d8-49df-8a2b-5b8fce224611 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:06.021634 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3cad93-53d8-49df-8a2b-5b8fce224611 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:06.022294 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3cad93-53d8-49df-8a2b-5b8fce224611 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:06.022776 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3cad93-53d8-49df-8a2b-5b8fce224611 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:06.024261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3cad93-53d8-49df-8a2b-5b8fce224611 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:06.030582 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10c16e-0261-4777-8515-48f517e614e7 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:06.033509 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 456/896] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Sep 10 23:43:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:06.035640 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3cad93-53d8-49df-8a2b-5b8fce224611 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:06.079863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d29c97-643a-4daa-a3a6-e0dd0ed87704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:06.080484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d29c97-643a-4daa-a3a6-e0dd0ed87704 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:06.080760 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d29c97-643a-4daa-a3a6-e0dd0ed87704 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:06.081034 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d29c97-643a-4daa-a3a6-e0dd0ed87704 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:06.081848 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d29c97-643a-4daa-a3a6-e0dd0ed87704 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:06.083284 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d29c97-643a-4daa-a3a6-e0dd0ed87704 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:06.143848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3cad93-53d8-49df-8a2b-5b8fce224611 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:06.154718 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 441/897] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Sep 10 23:43:05 2022] POST /identity/v3/services => generated 241 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:43:06.154718 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d29c97-643a-4daa-a3a6-e0dd0ed87704 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:06.209942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e350117-f6c7-402f-a615-88c728d6ac94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:06.211350 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e350117-f6c7-402f-a615-88c728d6ac94 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:06.211964 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e350117-f6c7-402f-a615-88c728d6ac94 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:06.212422 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e350117-f6c7-402f-a615-88c728d6ac94 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:06.229563 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e350117-f6c7-402f-a615-88c728d6ac94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:43:06.237074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-2e350117-f6c7-402f-a615-88c728d6ac94 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 23:43:06.237074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:06.237074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:06.237074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:06.237074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:06.237074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:06.237074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:06.237074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:06.237074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:06.237074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:06.237074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:06.237074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:06.237074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:43:06.237074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:43:06.237074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:43:06.237074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:43:06.238128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:43:06.238128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:43:06.238128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:43:06.238128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:43:06.238128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:06.238128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:06.238128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:06.238128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:06.238128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:06.238128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:06.238128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:06.238128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:06.238128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:06.238128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:06.238128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:06.238128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:06.239261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:06.239261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:06.239261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:06.239261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:06.239261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:43:06.239261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:43:06.239261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:43:06.239261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:43:06.239261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:43:06.239261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:43:06.239261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 23:43:06.239261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:06.243723 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 457/898] 10.222.0.17 () {56 vars in 1120 bytes} [Sat Sep 10 23:43:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:06.245489 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e350117-f6c7-402f-a615-88c728d6ac94 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:06.298214 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d29c97-643a-4daa-a3a6-e0dd0ed87704 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:06.314594 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:06.337602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:06.345202 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622412b5-468e-4f8d-90f8-a8425c8fe35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:06.346241 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622412b5-468e-4f8d-90f8-a8425c8fe35f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:06.346829 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622412b5-468e-4f8d-90f8-a8425c8fe35f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:06.347325 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622412b5-468e-4f8d-90f8-a8425c8fe35f None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:06.348236 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622412b5-468e-4f8d-90f8-a8425c8fe35f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:06.348865 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622412b5-468e-4f8d-90f8-a8425c8fe35f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:06.350606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622412b5-468e-4f8d-90f8-a8425c8fe35f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:06.364146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 442/899] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Sep 10 23:43:06 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:06.367904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622412b5-468e-4f8d-90f8-a8425c8fe35f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:06.392880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e350117-f6c7-402f-a615-88c728d6ac94 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:06.446181 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4b9f5-a215-41b4-b4c6-f5517fec8f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:06.446884 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4b9f5-a215-41b4-b4c6-f5517fec8f93 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:06.447160 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4b9f5-a215-41b4-b4c6-f5517fec8f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:06.447434 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4b9f5-a215-41b4-b4c6-f5517fec8f93 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:06.448607 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4b9f5-a215-41b4-b4c6-f5517fec8f93 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:06.448946 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4b9f5-a215-41b4-b4c6-f5517fec8f93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:06.450396 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4b9f5-a215-41b4-b4c6-f5517fec8f93 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:06.462305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 458/900] 10.222.0.17 () {56 vars in 1115 bytes} [Sat Sep 10 23:43:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:06.467944 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4b9f5-a215-41b4-b4c6-f5517fec8f93 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:06.512819 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622412b5-468e-4f8d-90f8-a8425c8fe35f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:06.563985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5ff5f-a9d4-4b4e-96e0-b48df824da06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:06.565169 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5ff5f-a9d4-4b4e-96e0-b48df824da06 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:06.565651 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5ff5f-a9d4-4b4e-96e0-b48df824da06 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:06.566341 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5ff5f-a9d4-4b4e-96e0-b48df824da06 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:06.567726 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e5ff5f-a9d4-4b4e-96e0-b48df824da06 None admin] RBAC: Adding query filter params (interface=public, service_id=9f58413f8ee84dc4a5b4a6f5929aeae8, region_id=RegionOne) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:06.568492 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e5ff5f-a9d4-4b4e-96e0-b48df824da06 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:06.570738 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e5ff5f-a9d4-4b4e-96e0-b48df824da06 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:06.583862 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 443/901] 10.222.0.17 () {56 vars in 1251 bytes} [Sat Sep 10 23:43:06 2022] GET /identity/v3/endpoints?service_id=9f58413f8ee84dc4a5b4a6f5929aeae8&interface=public®ion_id=RegionOne => generated 194 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:06.585615 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5ff5f-a9d4-4b4e-96e0-b48df824da06 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:06.595433 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4b9f5-a215-41b4-b4c6-f5517fec8f93 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:06.605970 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:06.645399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee09532-3aa8-4c77-a5c6-a20515ad4ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:06.646983 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee09532-3aa8-4c77-a5c6-a20515ad4ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:06.647475 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee09532-3aa8-4c77-a5c6-a20515ad4ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:06.648169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee09532-3aa8-4c77-a5c6-a20515ad4ae0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:06.653616 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:06.664633 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee09532-3aa8-4c77-a5c6-a20515ad4ae0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:06.668765 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee09532-3aa8-4c77-a5c6-a20515ad4ae0 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:06.688987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-1ee09532-3aa8-4c77-a5c6-a20515ad4ae0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:06.688987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:06.688987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:06.688987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:06.688987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:06.688987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:06.688987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:06.688987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:06.688987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:06.688987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:06.688987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:06.688987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:06.688987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:43:06.688987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:43:06.688987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:43:06.688987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:43:06.690170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:06.690170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:06.690170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:06.690170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:06.690170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:06.690170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:06.690170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:06.690170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:06.690170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:06.690170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:06.690170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:06.690170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:06.690170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:06.690170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:06.690170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:06.690170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:06.690170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:43:06.691272 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:43:06.691272 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:06.691272 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:06.698863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 459/902] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:43:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:43:06.700068 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee09532-3aa8-4c77-a5c6-a20515ad4ae0 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:06.703061 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5ff5f-a9d4-4b4e-96e0-b48df824da06 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:06.747261 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d813d8f6-cd91-425b-b03b-11b1e466bddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:06.748117 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813d8f6-cd91-425b-b03b-11b1e466bddb None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:06.748839 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813d8f6-cd91-425b-b03b-11b1e466bddb None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:06.749420 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813d8f6-cd91-425b-b03b-11b1e466bddb None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:06.750309 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d813d8f6-cd91-425b-b03b-11b1e466bddb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:06.752412 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d813d8f6-cd91-425b-b03b-11b1e466bddb None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:06.760448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 444/903] 10.222.0.17 () {56 vars in 1087 bytes} [Sat Sep 10 23:43:06 2022] GET /identity/v3/services => generated 1497 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 23:43:06.816509 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee09532-3aa8-4c77-a5c6-a20515ad4ae0 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:06.858285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc8a44-694f-40ac-aeb8-45c19d4f379d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:06.859248 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc8a44-694f-40ac-aeb8-45c19d4f379d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:06.859896 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc8a44-694f-40ac-aeb8-45c19d4f379d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:06.860326 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc8a44-694f-40ac-aeb8-45c19d4f379d None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:06.861180 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc8a44-694f-40ac-aeb8-45c19d4f379d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:06.861637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc8a44-694f-40ac-aeb8-45c19d4f379d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:06.863305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc8a44-694f-40ac-aeb8-45c19d4f379d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:06.877157 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 460/904] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:06.887836 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d813d8f6-cd91-425b-b03b-11b1e466bddb None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:07.002755 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d813d8f6-cd91-425b-b03b-11b1e466bddb None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:07.028510 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:07.040298 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9d1de-2e5d-4adc-900d-b9734d31815b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:07.041198 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9d1de-2e5d-4adc-900d-b9734d31815b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:07.041670 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9d1de-2e5d-4adc-900d-b9734d31815b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:07.042652 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9d1de-2e5d-4adc-900d-b9734d31815b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:07.050902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9d1de-2e5d-4adc-900d-b9734d31815b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:07.052587 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9d1de-2e5d-4adc-900d-b9734d31815b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:07.065273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-e5c9d1de-2e5d-4adc-900d-b9734d31815b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 23:43:07.065273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:07.065273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:07.065273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:07.065273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:07.065273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:07.065273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:07.065273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:07.065273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:07.065273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:07.065273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:07.065273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:07.065273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:43:07.065273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:43:07.065273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:43:07.065273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:43:07.066219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:07.066219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:07.066219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:43:07.066219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:43:07.066219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:43:07.066219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:43:07.066219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:07.066219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:07.066219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:07.066219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:07.066219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:07.066219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:07.066219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:07.066219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:07.066219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:07.066219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:07.066219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:07.066219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:07.067205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:07.067205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:07.067205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:43:07.067205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:43:07.067205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:43:07.067205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:43:07.067205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:43:07.067205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:43:07.067205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 23:43:07.067205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:07.074861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 445/905] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Sep 10 23:43:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:07.076443 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc8a44-694f-40ac-aeb8-45c19d4f379d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:07.097353 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:07.186973 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc8a44-694f-40ac-aeb8-45c19d4f379d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:07.223114 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33464e1c-7b9a-4db9-8be9-a26ad7fc75f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:07.224600 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33464e1c-7b9a-4db9-8be9-a26ad7fc75f6 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:07.225070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33464e1c-7b9a-4db9-8be9-a26ad7fc75f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:07.225530 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33464e1c-7b9a-4db9-8be9-a26ad7fc75f6 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:07.226531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33464e1c-7b9a-4db9-8be9-a26ad7fc75f6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:07.227056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33464e1c-7b9a-4db9-8be9-a26ad7fc75f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:07.228683 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33464e1c-7b9a-4db9-8be9-a26ad7fc75f6 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:07.250794 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 461/906] 10.222.0.17 () {56 vars in 1149 bytes} [Sat Sep 10 23:43:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:07.253362 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9d1de-2e5d-4adc-900d-b9734d31815b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:07.360625 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9d1de-2e5d-4adc-900d-b9734d31815b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:07.396426 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae9aee-ee78-4001-b418-fd8bb8f20672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:07.397028 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae9aee-ee78-4001-b418-fd8bb8f20672 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:07.397307 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae9aee-ee78-4001-b418-fd8bb8f20672 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:07.397580 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae9aee-ee78-4001-b418-fd8bb8f20672 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:07.452060 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:07.472579 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae9aee-ee78-4001-b418-fd8bb8f20672 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:07.474353 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae9aee-ee78-4001-b418-fd8bb8f20672 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:07.587030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 446/907] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:43:07 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 23:43:07.588998 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33464e1c-7b9a-4db9-8be9-a26ad7fc75f6 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:07.719434 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:07.741769 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33464e1c-7b9a-4db9-8be9-a26ad7fc75f6 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:07.786715 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d534a2b8-a4ff-4f32-9413-4b636684c2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:07.786715 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534a2b8-a4ff-4f32-9413-4b636684c2fb None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:07.786715 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534a2b8-a4ff-4f32-9413-4b636684c2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:07.786715 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534a2b8-a4ff-4f32-9413-4b636684c2fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:07.797168 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d534a2b8-a4ff-4f32-9413-4b636684c2fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:07.798644 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d534a2b8-a4ff-4f32-9413-4b636684c2fb None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:07.808287 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4118bbd-13ef-436a-8198-a0f2167d751c None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:07.810692 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4118bbd-13ef-436a-8198-a0f2167d751c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:07.810692 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4118bbd-13ef-436a-8198-a0f2167d751c None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:07.814475 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 447/908] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:07.815666 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f1df1-ef46-488e-bad5-7e26628a89ce None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:07.815946 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f1df1-ef46-488e-bad5-7e26628a89ce None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:07.816218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f1df1-ef46-488e-bad5-7e26628a89ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:07.816757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-d534a2b8-a4ff-4f32-9413-4b636684c2fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:07.816757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:07.816757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:07.816757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:07.816757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:07.816757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:07.816757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:07.816757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:07.816757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:07.816757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:07.816757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:07.816757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:07.816757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:43:07.816757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:43:07.816757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:43:07.816757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:43:07.817664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:07.817664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:07.817664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:07.817664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:07.817664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:07.817664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:07.817664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:07.817664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:07.817664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:07.817664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:07.817664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:07.817664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:07.817664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:07.817664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:07.817664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:07.817664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:07.817664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:43:07.819164 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:43:07.819164 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:07.819164 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:07.819164 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 462/909] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:43:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:07.825637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d534a2b8-a4ff-4f32-9413-4b636684c2fb None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:07.856019 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b85f1df1-ef46-488e-bad5-7e26628a89ce None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:07.857290 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b85f1df1-ef46-488e-bad5-7e26628a89ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:07.942409 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d534a2b8-a4ff-4f32-9413-4b636684c2fb None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:07.971505 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:07.980383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956ea7d-8605-464a-b512-47f6b13f8aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:07.980990 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956ea7d-8605-464a-b512-47f6b13f8aed None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:07.981270 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956ea7d-8605-464a-b512-47f6b13f8aed None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:07.981542 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956ea7d-8605-464a-b512-47f6b13f8aed None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:07.982180 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f956ea7d-8605-464a-b512-47f6b13f8aed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:07.982486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f956ea7d-8605-464a-b512-47f6b13f8aed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:07.984268 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f956ea7d-8605-464a-b512-47f6b13f8aed None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:07.999448 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 463/910] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:08.004265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 448/911] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:43:08.015727 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956ea7d-8605-464a-b512-47f6b13f8aed None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:08.118806 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956ea7d-8605-464a-b512-47f6b13f8aed None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:08.154953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b07fc-37c8-45fd-b6db-7741ad0e17fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:08.155589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b07fc-37c8-45fd-b6db-7741ad0e17fc None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:08.155893 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b07fc-37c8-45fd-b6db-7741ad0e17fc None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:08.156155 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b07fc-37c8-45fd-b6db-7741ad0e17fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:08.163255 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b07fc-37c8-45fd-b6db-7741ad0e17fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:08.164760 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b07fc-37c8-45fd-b6db-7741ad0e17fc None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:08.176967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-983b07fc-37c8-45fd-b6db-7741ad0e17fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:43:08.176967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:08.176967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:08.176967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:08.176967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:08.176967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:08.176967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:08.176967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:08.176967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:08.176967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:08.176967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:08.176967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:08.176967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:43:08.176967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:43:08.176967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:43:08.177948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:43:08.177948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:08.177948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:08.177948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:08.177948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:08.177948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:08.177948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:08.177948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:08.177948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:08.177948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:08.177948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:08.177948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:08.177948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:08.177948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:08.177948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:08.177948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:08.177948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:08.179237 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:43:08.179237 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:43:08.179237 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:43:08.179237 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:43:08.179237 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:43:08.179237 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:43:08.179237 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:43:08.179237 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:08.179237 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 464/912] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:43:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:08.186440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f1df1-ef46-488e-bad5-7e26628a89ce None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:08.204127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:08.319074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f1df1-ef46-488e-bad5-7e26628a89ce None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:08.355857 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b539b1d2-b74f-4a69-bed5-d3021b41b1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:08.356911 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539b1d2-b74f-4a69-bed5-d3021b41b1c9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:08.357193 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539b1d2-b74f-4a69-bed5-d3021b41b1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:08.357456 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539b1d2-b74f-4a69-bed5-d3021b41b1c9 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:08.357925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b07fc-37c8-45fd-b6db-7741ad0e17fc None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:08.358368 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b539b1d2-b74f-4a69-bed5-d3021b41b1c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:08.358446 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b539b1d2-b74f-4a69-bed5-d3021b41b1c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:08.359876 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b539b1d2-b74f-4a69-bed5-d3021b41b1c9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:08.376776 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 449/913] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:43:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:08.377985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b539b1d2-b74f-4a69-bed5-d3021b41b1c9 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:08.419971 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:08.461363 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:08.487879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b07fc-37c8-45fd-b6db-7741ad0e17fc None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:08.488920 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b539b1d2-b74f-4a69-bed5-d3021b41b1c9 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:08.530632 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34422ba1-e0ac-408e-9dd2-12e60c39ea95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:08.531625 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34422ba1-e0ac-408e-9dd2-12e60c39ea95 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:08.532122 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34422ba1-e0ac-408e-9dd2-12e60c39ea95 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:08.532600 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34422ba1-e0ac-408e-9dd2-12e60c39ea95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:08.532705 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad517c4f-6bc0-4a95-b330-e95470322eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:08.533262 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad517c4f-6bc0-4a95-b330-e95470322eef None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:08.533537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad517c4f-6bc0-4a95-b330-e95470322eef None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:08.533799 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad517c4f-6bc0-4a95-b330-e95470322eef None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:08.534284 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34422ba1-e0ac-408e-9dd2-12e60c39ea95 None admin] RBAC: Adding query filter params (role.id=24493ac356254954854aeec4ad5717d5, scope.project.id=947277c4458645899287e6818947c076, user.id=b0a879085a194b0687e1cdcbafd48033) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:08.534390 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad517c4f-6bc0-4a95-b330-e95470322eef None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:08.534913 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34422ba1-e0ac-408e-9dd2-12e60c39ea95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:08.535793 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad517c4f-6bc0-4a95-b330-e95470322eef None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:08.536777 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34422ba1-e0ac-408e-9dd2-12e60c39ea95 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:08.544853 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 450/914] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:43:08 2022] GET /identity/v3/role_assignments?user.id=b0a879085a194b0687e1cdcbafd48033&scope.project.id=947277c4458645899287e6818947c076&role.id=24493ac356254954854aeec4ad5717d5 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:08.551776 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-ad517c4f-6bc0-4a95-b330-e95470322eef None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 23:43:08.551776 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:08.551776 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 23:43:08.551776 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 23:43:08.551776 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 23:43:08.551776 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 23:43:08.551776 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 23:43:08.551776 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:08.551776 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 23:43:08.551776 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:08.551776 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:43:08.551776 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:08.551776 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:08.551776 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:08.551776 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:08.551776 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:08.551776 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:08.552818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:08.552818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:08.552818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:08.552818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:08.552818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:08.552818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:08.552818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 23:43:08.552818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 23:43:08.552818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 23:43:08.552818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 23:43:08.552818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:08.552818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:08.552818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:08.552818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:08.552818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:08.552818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:08.553654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:08.553654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:08.553654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:08.553654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:08.553654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:08.553654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:08.553654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:08.553654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:08.553654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:08.553654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:08.553654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 23:43:08.553654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:08.553654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 23:43:08.553654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:08.553654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 465/915] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Sep 10 23:43:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:08.561639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34422ba1-e0ac-408e-9dd2-12e60c39ea95 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:08.564720 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79244f68-bb1d-4e6b-b7ae-e22baf315930 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:08.564996 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79244f68-bb1d-4e6b-b7ae-e22baf315930 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:08.565274 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79244f68-bb1d-4e6b-b7ae-e22baf315930 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:08.567686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 466/916] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:43:08.572946 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67841e4d-39da-4817-b868-54b109de5ae9 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:08.573864 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67841e4d-39da-4817-b868-54b109de5ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:08.574040 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67841e4d-39da-4817-b868-54b109de5ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:08.593818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-67841e4d-39da-4817-b868-54b109de5ae9 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:08.595110 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-67841e4d-39da-4817-b868-54b109de5ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:08.686790 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34422ba1-e0ac-408e-9dd2-12e60c39ea95 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:08.696968 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 467/917] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:43:08.733465 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622a475-1311-4a29-95df-a1102249dc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:08.734074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622a475-1311-4a29-95df-a1102249dc50 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:08.734353 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622a475-1311-4a29-95df-a1102249dc50 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:08.734622 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622a475-1311-4a29-95df-a1102249dc50 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:08.735302 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622a475-1311-4a29-95df-a1102249dc50 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:08.735600 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622a475-1311-4a29-95df-a1102249dc50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:08.736991 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622a475-1311-4a29-95df-a1102249dc50 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:08.742078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 451/918] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Sep 10 23:43:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:08.748139 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67841e4d-39da-4817-b868-54b109de5ae9 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:08.794176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:08.852808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67841e4d-39da-4817-b868-54b109de5ae9 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:08.892583 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b908e06-f02d-4d85-b607-c7070d7ced4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:08.893196 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b908e06-f02d-4d85-b607-c7070d7ced4a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:08.893474 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b908e06-f02d-4d85-b607-c7070d7ced4a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:08.893746 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b908e06-f02d-4d85-b607-c7070d7ced4a None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:08.894375 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b908e06-f02d-4d85-b607-c7070d7ced4a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:08.894685 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b908e06-f02d-4d85-b607-c7070d7ced4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:08.896063 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b908e06-f02d-4d85-b607-c7070d7ced4a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:08.901799 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 468/919] 10.222.0.17 () {56 vars in 1114 bytes} [Sat Sep 10 23:43:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:08.907515 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622a475-1311-4a29-95df-a1102249dc50 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:08.913213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:08.974396 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b908e06-f02d-4d85-b607-c7070d7ced4a None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:09.018056 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622a475-1311-4a29-95df-a1102249dc50 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:09.055192 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0a35d-8a32-41c2-970f-ffc87e9a60ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:09.055793 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0a35d-8a32-41c2-970f-ffc87e9a60ea None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:09.056073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0a35d-8a32-41c2-970f-ffc87e9a60ea None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:09.056346 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0a35d-8a32-41c2-970f-ffc87e9a60ea None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:09.057085 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd0a35d-8a32-41c2-970f-ffc87e9a60ea None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:09.059374 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd0a35d-8a32-41c2-970f-ffc87e9a60ea None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:09.082347 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b908e06-f02d-4d85-b607-c7070d7ced4a None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:09.090459 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:09.125937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b91c4-2c62-4d96-b243-bafa091180fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:09.126592 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b91c4-2c62-4d96-b243-bafa091180fd None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:09.127058 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b91c4-2c62-4d96-b243-bafa091180fd None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:09.127336 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b91c4-2c62-4d96-b243-bafa091180fd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:09.127983 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214b91c4-2c62-4d96-b243-bafa091180fd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:09.129909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214b91c4-2c62-4d96-b243-bafa091180fd None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:09.142571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-214b91c4-2c62-4d96-b243-bafa091180fd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 23:43:09.142571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:09.142571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 23:43:09.142571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 23:43:09.142571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 23:43:09.142571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 23:43:09.142571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 23:43:09.142571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:09.142571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 23:43:09.142571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:09.142571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:43:09.142571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:09.142571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:09.142571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:09.142571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:09.142571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:09.142571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:09.143627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:09.143627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:09.143627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:09.143627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:09.143627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:09.143627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:09.143627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 23:43:09.143627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 23:43:09.143627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 23:43:09.143627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 23:43:09.143627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:09.143627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:09.143627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:09.143627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:09.143627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:09.143627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:09.144885 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:09.144885 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:09.144885 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:09.144885 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:09.144885 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:09.144885 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:09.144885 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:09.144885 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:09.144885 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:09.144885 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:09.144885 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 23:43:09.144885 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:09.144885 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 23:43:09.144885 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:09.144885 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 469/921] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Sep 10 23:43:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:43:09.146315 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 452/921] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Sep 10 23:43:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:43:09.149693 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b91c4-2c62-4d96-b243-bafa091180fd None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:09.233099 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6368e6-fab3-4033-be15-ba4c445d908b None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:09.233291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6368e6-fab3-4033-be15-ba4c445d908b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:09.233546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6368e6-fab3-4033-be15-ba4c445d908b None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:09.235068 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 453/922] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:43:09.244650 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5072a-92b2-4d4a-9bbc-dd931236d6fe None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:09.245713 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5072a-92b2-4d4a-9bbc-dd931236d6fe None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:09.246079 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5072a-92b2-4d4a-9bbc-dd931236d6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:09.273891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1cd5072a-92b2-4d4a-9bbc-dd931236d6fe None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:09.275235 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1cd5072a-92b2-4d4a-9bbc-dd931236d6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:09.275905 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b91c4-2c62-4d96-b243-bafa091180fd None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:09.341434 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac801259-ae51-4501-907a-3812e7c77824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:09.341992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac801259-ae51-4501-907a-3812e7c77824 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:09.342395 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac801259-ae51-4501-907a-3812e7c77824 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:09.342784 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac801259-ae51-4501-907a-3812e7c77824 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:09.343687 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac801259-ae51-4501-907a-3812e7c77824 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:09.343986 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac801259-ae51-4501-907a-3812e7c77824 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:09.345441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac801259-ae51-4501-907a-3812e7c77824 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:09.351820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 470/923] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Sep 10 23:43:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:09.357090 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:09.358996 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac801259-ae51-4501-907a-3812e7c77824 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:09.379531 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 454/924] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 23:43:09.414047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:09.487116 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac801259-ae51-4501-907a-3812e7c77824 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:09.521279 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9464c6-7aaf-47cd-8036-5e533b682cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:09.521903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9464c6-7aaf-47cd-8036-5e533b682cce None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:09.522175 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9464c6-7aaf-47cd-8036-5e533b682cce None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:09.522444 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9464c6-7aaf-47cd-8036-5e533b682cce None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:09.523139 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9464c6-7aaf-47cd-8036-5e533b682cce None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:09.523435 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9464c6-7aaf-47cd-8036-5e533b682cce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:09.524915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9464c6-7aaf-47cd-8036-5e533b682cce None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:09.531751 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 471/925] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Sep 10 23:43:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:09.537617 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5072a-92b2-4d4a-9bbc-dd931236d6fe None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:09.641112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5072a-92b2-4d4a-9bbc-dd931236d6fe None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:09.675607 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516fd5c-8714-4c52-843e-b7ebaf67d854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:09.676269 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516fd5c-8714-4c52-843e-b7ebaf67d854 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:09.676542 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516fd5c-8714-4c52-843e-b7ebaf67d854 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:09.676859 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516fd5c-8714-4c52-843e-b7ebaf67d854 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:09.677559 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516fd5c-8714-4c52-843e-b7ebaf67d854 None admin] RBAC: Adding query filter params (interface=public, service_id=10fea1fdca244d1887b4641c3389e730, region_id=RegionOne) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:09.677862 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516fd5c-8714-4c52-843e-b7ebaf67d854 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:09.679313 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516fd5c-8714-4c52-843e-b7ebaf67d854 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:09.685760 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 455/926] 10.222.0.17 () {56 vars in 1251 bytes} [Sat Sep 10 23:43:09 2022] GET /identity/v3/endpoints?service_id=10fea1fdca244d1887b4641c3389e730&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:09.691531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9464c6-7aaf-47cd-8036-5e533b682cce None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:09.693432 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516fd5c-8714-4c52-843e-b7ebaf67d854 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:09.764281 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:09.818865 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:09.828266 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9464c6-7aaf-47cd-8036-5e533b682cce None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:09.831643 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516fd5c-8714-4c52-843e-b7ebaf67d854 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:09.874824 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d922ecf-1308-4fdd-8f5f-1944c43a9cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:09.875040 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46193be-8ef0-4b11-b066-4f7161bad484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:09.875649 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46193be-8ef0-4b11-b066-4f7161bad484 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:09.875925 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46193be-8ef0-4b11-b066-4f7161bad484 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:09.876180 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d922ecf-1308-4fdd-8f5f-1944c43a9cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:09.876299 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46193be-8ef0-4b11-b066-4f7161bad484 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:09.876755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d922ecf-1308-4fdd-8f5f-1944c43a9cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:09.876867 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46193be-8ef0-4b11-b066-4f7161bad484 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:09.877270 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d922ecf-1308-4fdd-8f5f-1944c43a9cc4 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:09.878020 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d922ecf-1308-4fdd-8f5f-1944c43a9cc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:09.878139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46193be-8ef0-4b11-b066-4f7161bad484 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:09.879803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d922ecf-1308-4fdd-8f5f-1944c43a9cc4 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:09.888863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-f46193be-8ef0-4b11-b066-4f7161bad484 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 23:43:09.888863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:09.888863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 23:43:09.888863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 23:43:09.888863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 23:43:09.888863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 23:43:09.888863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 23:43:09.888863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:09.888863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 23:43:09.888863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:09.888863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:43:09.888863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:09.888863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:09.888863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:09.888863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:09.888863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:09.890021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:09.890021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:09.890021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:09.890021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:09.890021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:09.890021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:09.890021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:09.890021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 23:43:09.890021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 23:43:09.890021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 23:43:09.890021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 23:43:09.890021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:09.890021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:09.890021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:09.890021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:09.890021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:09.890021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:09.891158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:09.891158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:09.891158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:09.891158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:09.891158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:09.891158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:09.891158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:09.891158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:09.891158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:09.891158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:09.891158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 23:43:09.891158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:09.891158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 23:43:09.891158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:09.898427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 456/927] 10.222.0.17 () {56 vars in 1132 bytes} [Sat Sep 10 23:43:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:09.899410 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 472/928] 10.222.0.17 () {56 vars in 1087 bytes} [Sat Sep 10 23:43:09 2022] GET /identity/v3/services => generated 1497 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 23:43:09.900443 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46193be-8ef0-4b11-b066-4f7161bad484 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:10.015381 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46193be-8ef0-4b11-b066-4f7161bad484 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:10.035366 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:10.053867 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f71d3-e50f-4702-93cf-6870a8996f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:10.054476 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f71d3-e50f-4702-93cf-6870a8996f52 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:10.057558 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f71d3-e50f-4702-93cf-6870a8996f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:10.057558 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f71d3-e50f-4702-93cf-6870a8996f52 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:10.057558 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f71d3-e50f-4702-93cf-6870a8996f52 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:10.057558 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f71d3-e50f-4702-93cf-6870a8996f52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:10.057922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f71d3-e50f-4702-93cf-6870a8996f52 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:10.063494 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 457/929] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Sep 10 23:43:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:10.075821 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d922ecf-1308-4fdd-8f5f-1944c43a9cc4 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:10.191897 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d922ecf-1308-4fdd-8f5f-1944c43a9cc4 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:10.226448 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb5376-7c41-4ff1-b1ec-be5ce79c4b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:10.227123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb5376-7c41-4ff1-b1ec-be5ce79c4b8b None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:10.227400 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb5376-7c41-4ff1-b1ec-be5ce79c4b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:10.227665 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb5376-7c41-4ff1-b1ec-be5ce79c4b8b None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:10.228288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb5376-7c41-4ff1-b1ec-be5ce79c4b8b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:10.228582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb5376-7c41-4ff1-b1ec-be5ce79c4b8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:10.230384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb5376-7c41-4ff1-b1ec-be5ce79c4b8b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:10.236061 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 473/930] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Sep 10 23:43:10 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:10.241694 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f71d3-e50f-4702-93cf-6870a8996f52 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:10.345974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f71d3-e50f-4702-93cf-6870a8996f52 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:10.380377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0be770-315a-44c1-98e4-07cf3b3d2117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:10.381002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0be770-315a-44c1-98e4-07cf3b3d2117 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:10.381233 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0be770-315a-44c1-98e4-07cf3b3d2117 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:10.381497 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0be770-315a-44c1-98e4-07cf3b3d2117 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:10.382165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0be770-315a-44c1-98e4-07cf3b3d2117 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:10.383555 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0be770-315a-44c1-98e4-07cf3b3d2117 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:10.479051 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 458/931] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Sep 10 23:43:10 2022] POST /identity/v3/endpoints => generated 347 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:43:10.897313 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f1306-da5e-478b-bfd2-1934ef157a76 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:10.897843 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f1306-da5e-478b-bfd2-1934ef157a76 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:10.897843 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f1306-da5e-478b-bfd2-1934ef157a76 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:10.899551 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 474/932] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:10.905524 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b81ef5-2864-489e-b27f-7f25f5a67524 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:10.905811 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b81ef5-2864-489e-b27f-7f25f5a67524 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:10.906082 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b81ef5-2864-489e-b27f-7f25f5a67524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:10.925765 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c4b81ef5-2864-489e-b27f-7f25f5a67524 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:10.926965 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c4b81ef5-2864-489e-b27f-7f25f5a67524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:10.956509 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:11.002610 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 459/933] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 23:43:11.273478 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f1306-da5e-478b-bfd2-1934ef157a76 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:11.306734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:11.390802 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f1306-da5e-478b-bfd2-1934ef157a76 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:11.430103 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424cdba-62a9-44fa-905e-008050834fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:11.431134 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424cdba-62a9-44fa-905e-008050834fbc None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:11.431570 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424cdba-62a9-44fa-905e-008050834fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:11.431992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424cdba-62a9-44fa-905e-008050834fbc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:11.443452 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424cdba-62a9-44fa-905e-008050834fbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:43:11.448920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-9424cdba-62a9-44fa-905e-008050834fbc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 23:43:11.448920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:11.448920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:11.448920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:11.448920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:11.448920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:11.448920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:11.448920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:11.448920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:11.448920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:11.448920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:11.448920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:11.448920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:43:11.448920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:43:11.448920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:43:11.448920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:43:11.449989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:43:11.449989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:43:11.449989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:43:11.449989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:43:11.449989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:11.449989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:11.449989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:11.449989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:11.449989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:11.449989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:11.449989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:11.449989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:11.449989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:11.449989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:11.449989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:11.449989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:11.451027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:11.451027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:11.451027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:11.451027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:11.451027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:43:11.451027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:43:11.451027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:43:11.451027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:43:11.451027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:43:11.451027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:43:11.451027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 23:43:11.451027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:11.453271 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 475/934] 10.222.0.17 () {56 vars in 1120 bytes} [Sat Sep 10 23:43:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:11.459225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b81ef5-2864-489e-b27f-7f25f5a67524 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:11.497665 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c24439-909a-4ba5-9324-ceb35b7b87b8 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:11.497944 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c24439-909a-4ba5-9324-ceb35b7b87b8 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:11.498221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c24439-909a-4ba5-9324-ceb35b7b87b8 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:11.499844 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 476/935] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:43:11.504787 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5504c-d015-4659-8775-e395b8a4ae42 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:11.505003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5504c-d015-4659-8775-e395b8a4ae42 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:11.505324 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5504c-d015-4659-8775-e395b8a4ae42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:11.523755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-45a5504c-d015-4659-8775-e395b8a4ae42 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:11.524836 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-45a5504c-d015-4659-8775-e395b8a4ae42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:11.567110 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b81ef5-2864-489e-b27f-7f25f5a67524 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:11.599101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 477/936] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:43:11.601782 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f15bf-6f9b-4cbb-a0df-562122cc9a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:11.602607 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f15bf-6f9b-4cbb-a0df-562122cc9a98 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:11.603176 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f15bf-6f9b-4cbb-a0df-562122cc9a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:11.603638 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f15bf-6f9b-4cbb-a0df-562122cc9a98 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:11.605431 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8f15bf-6f9b-4cbb-a0df-562122cc9a98 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:11.605944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8f15bf-6f9b-4cbb-a0df-562122cc9a98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:11.607533 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8f15bf-6f9b-4cbb-a0df-562122cc9a98 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:11.616555 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 460/937] 10.222.0.17 () {56 vars in 1115 bytes} [Sat Sep 10 23:43:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:11.622492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5504c-d015-4659-8775-e395b8a4ae42 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:11.655052 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:11.729048 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5504c-d015-4659-8775-e395b8a4ae42 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:11.764043 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c16e1-c489-4313-9f16-bd861c1c0d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:11.764672 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c16e1-c489-4313-9f16-bd861c1c0d25 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:11.764946 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c16e1-c489-4313-9f16-bd861c1c0d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:11.765212 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c16e1-c489-4313-9f16-bd861c1c0d25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:11.775641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c16e1-c489-4313-9f16-bd861c1c0d25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:11.777548 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c16e1-c489-4313-9f16-bd861c1c0d25 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:11.792689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-e91c16e1-c489-4313-9f16-bd861c1c0d25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:11.792689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:11.792689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:11.792689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:11.792689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:11.792689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:11.792689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:11.792689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:11.792689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:11.792689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:11.792689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:11.792689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:11.792689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:43:11.792689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:43:11.792689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:43:11.792689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:43:11.793663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:11.793663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:11.793663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:11.793663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:11.793663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:11.793663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:11.793663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:11.793663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:11.793663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:11.793663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:11.793663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:11.793663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:11.793663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:11.793663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:11.793663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:11.793663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:11.793663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:43:11.794528 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:43:11.794528 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:11.794528 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:11.794528 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 478/938] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:43:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:11.800900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f15bf-6f9b-4cbb-a0df-562122cc9a98 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:11.829128 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:11.887558 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c16e1-c489-4313-9f16-bd861c1c0d25 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:11.908203 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f15bf-6f9b-4cbb-a0df-562122cc9a98 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:11.944793 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e8187-3004-410d-80d4-3fafb46f353b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:11.945723 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e8187-3004-410d-80d4-3fafb46f353b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:11.946213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e8187-3004-410d-80d4-3fafb46f353b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:11.946694 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e8187-3004-410d-80d4-3fafb46f353b None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:11.947534 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e8187-3004-410d-80d4-3fafb46f353b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:11.948148 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e8187-3004-410d-80d4-3fafb46f353b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:11.949808 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e8187-3004-410d-80d4-3fafb46f353b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:11.955085 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:11.962399 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 461/939] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:11.968710 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e8187-3004-410d-80d4-3fafb46f353b None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:11.996627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c16e1-c489-4313-9f16-bd861c1c0d25 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:12.033667 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9d062-3a08-4360-b5e0-6d363d085ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:12.034221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9d062-3a08-4360-b5e0-6d363d085ae1 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:12.034603 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9d062-3a08-4360-b5e0-6d363d085ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:12.034906 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9d062-3a08-4360-b5e0-6d363d085ae1 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:12.035613 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e9d062-3a08-4360-b5e0-6d363d085ae1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:12.037043 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e9d062-3a08-4360-b5e0-6d363d085ae1 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:12.039949 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: WARNING keystone.server.flask.common [None req-84e9d062-3a08-4360-b5e0-6d363d085ae1 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 10 23:43:12.058855 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:12.075926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e8187-3004-410d-80d4-3fafb46f353b None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:12.088688 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.sql.core [None req-84e9d062-3a08-4360-b5e0-6d363d085ae1 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 10 23:43:12.088688 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 10 23:43:12.088688 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [parameters: {'id': '7682b0bdf75349d5a6e87b64b04533ff', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 10 23:43:12.088688 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81573) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 10 23:43:12.096446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-84e9d062-3a08-4360-b5e0-6d363d085ae1 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7682b0bdf75349d5a6e87b64b04533ff.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7682b0bdf75349d5a6e87b64b04533ff. Sep 10 23:43:12.096446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:12.096446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 23:43:12.096446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 23:43:12.096446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 23:43:12.096446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 23:43:12.096446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 23:43:12.096446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 23:43:12.096446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 23:43:12.096446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application conn.query(q) Sep 10 23:43:12.096446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 23:43:12.096446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 23:43:12.096446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 23:43:12.098565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application result.read() Sep 10 23:43:12.098565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 23:43:12.098565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 23:43:12.098565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 23:43:12.098565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 23:43:12.098565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 23:43:12.098565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 23:43:12.098565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 23:43:12.098565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 23:43:12.098565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 10 23:43:12.098565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:12.098565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 10 23:43:12.098565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:12.098565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:12.098565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 10 23:43:12.098565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 10 23:43:12.098565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 10 23:43:12.098565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 10 23:43:12.099759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 23:43:12.099759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application next(self.gen) Sep 10 23:43:12.099759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 10 23:43:12.099759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application yield resource Sep 10 23:43:12.099759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 23:43:12.099759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application next(self.gen) Sep 10 23:43:12.099759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 10 23:43:12.099759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self.session.rollback() Sep 10 23:43:12.099759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 10 23:43:12.099759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self.force_reraise() Sep 10 23:43:12.099759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 10 23:43:12.099759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise self.value Sep 10 23:43:12.099759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 10 23:43:12.099759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 10 23:43:12.099759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 10 23:43:12.099759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application session.commit() Sep 10 23:43:12.099759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 10 23:43:12.100645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 10 23:43:12.100645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 10 23:43:12.100645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._prepare_impl() Sep 10 23:43:12.100645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 10 23:43:12.100645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self.session.flush() Sep 10 23:43:12.100645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 10 23:43:12.100645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._flush(objects) Sep 10 23:43:12.100645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 10 23:43:12.100645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 10 23:43:12.100645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 10 23:43:12.100645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application compat.raise_( Sep 10 23:43:12.100645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 23:43:12.100645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception Sep 10 23:43:12.100645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 10 23:43:12.100645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application flush_context.execute() Sep 10 23:43:12.100645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 10 23:43:12.100645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rec.execute(self) Sep 10 23:43:12.101535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 10 23:43:12.101535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 10 23:43:12.101535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 10 23:43:12.101535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 10 23:43:12.101535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 10 23:43:12.101535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 10 23:43:12.101535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 10 23:43:12.101535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 10 23:43:12.101535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 10 23:43:12.101535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 10 23:43:12.101535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 10 23:43:12.101535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 10 23:43:12.101535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 10 23:43:12.101535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 10 23:43:12.101535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 10 23:43:12.102328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 10 23:43:12.102328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 23:43:12.102328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception Sep 10 23:43:12.102328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 23:43:12.102328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 23:43:12.102328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 23:43:12.102328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 23:43:12.102328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 23:43:12.102328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 23:43:12.102328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 23:43:12.102328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application conn.query(q) Sep 10 23:43:12.102328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 23:43:12.102328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 23:43:12.102328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 23:43:12.102328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application result.read() Sep 10 23:43:12.102328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 23:43:12.102328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 23:43:12.105940 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 23:43:12.105940 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 23:43:12.105940 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 23:43:12.105940 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 23:43:12.105940 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 23:43:12.105940 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 23:43:12.105940 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 10 23:43:12.105940 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 10 23:43:12.105940 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [parameters: {'id': '7682b0bdf75349d5a6e87b64b04533ff', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 10 23:43:12.105940 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 10 23:43:12.105940 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:12.105940 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:43:12.105940 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:12.105940 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:12.105940 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 10 23:43:12.107999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 10 23:43:12.107999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 10 23:43:12.107999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 10 23:43:12.107999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 10 23:43:12.107999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:12.107999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:43:12.107999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:12.107999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:12.107999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:12.107999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:12.107999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:12.107999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:12.107999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:12.107999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:12.107999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:12.107999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:12.107999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:12.110113 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:12.110113 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 10 23:43:12.110113 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 10 23:43:12.110113 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:12.110113 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:12.110113 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 10 23:43:12.110113 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 10 23:43:12.110113 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7682b0bdf75349d5a6e87b64b04533ff. Sep 10 23:43:12.110113 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:12.110113 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 479/940] 10.222.0.17 () {60 vars in 1140 bytes} [Sat Sep 10 23:43:11 2022] POST /identity/v3/projects => generated 262 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:12.110113 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9d062-3a08-4360-b5e0-6d363d085ae1 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:12.123855 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766f0ec-4c60-4b0a-9d5e-33887314c0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:12.124910 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766f0ec-4c60-4b0a-9d5e-33887314c0fa None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:12.125393 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766f0ec-4c60-4b0a-9d5e-33887314c0fa None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:12.125883 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766f0ec-4c60-4b0a-9d5e-33887314c0fa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:12.137832 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766f0ec-4c60-4b0a-9d5e-33887314c0fa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:12.139808 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766f0ec-4c60-4b0a-9d5e-33887314c0fa None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:12.157001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-a766f0ec-4c60-4b0a-9d5e-33887314c0fa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 23:43:12.157001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:12.157001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:12.157001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:12.157001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:12.157001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:12.157001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:12.157001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:12.157001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:12.157001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:12.157001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:12.157001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:12.157001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:43:12.157001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:43:12.157001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:43:12.157001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:43:12.158005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:12.158005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:12.158005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:43:12.158005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:43:12.158005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:43:12.158005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:43:12.158005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:12.158005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:12.158005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:12.158005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:12.158005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:12.158005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:12.158005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:12.158005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:12.158005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:12.158005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:12.158005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:12.158005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:12.159009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:12.159009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:12.159009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:43:12.159009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:43:12.159009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:43:12.159009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:43:12.159009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:43:12.159009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:43:12.159009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 23:43:12.159009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:12.162745 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 462/941] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Sep 10 23:43:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:12.165698 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766f0ec-4c60-4b0a-9d5e-33887314c0fa None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:12.237428 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9d062-3a08-4360-b5e0-6d363d085ae1 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:12.261475 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:12.276804 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2178c9-b7e8-40c1-add5-895a16ae9a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:12.277589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2178c9-b7e8-40c1-add5-895a16ae9a8f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:12.277983 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2178c9-b7e8-40c1-add5-895a16ae9a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:12.278324 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2178c9-b7e8-40c1-add5-895a16ae9a8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:12.285449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2178c9-b7e8-40c1-add5-895a16ae9a8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:12.287371 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2178c9-b7e8-40c1-add5-895a16ae9a8f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:12.288400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766f0ec-4c60-4b0a-9d5e-33887314c0fa None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:12.303874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-de2178c9-b7e8-40c1-add5-895a16ae9a8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:43:12.303874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:12.303874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:12.303874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:12.303874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:12.303874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:12.303874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:12.303874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:12.303874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:12.303874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:12.303874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:12.303874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:12.303874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:43:12.303874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:43:12.303874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:43:12.305091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:43:12.305091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:12.305091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:12.305091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:12.305091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:12.305091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:12.305091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:12.305091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:12.305091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:12.305091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:12.305091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:12.305091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:12.305091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:12.305091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:12.305091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:12.305091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:12.305091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:12.306537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:43:12.306537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:43:12.306537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:43:12.306537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:43:12.306537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:43:12.306537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:43:12.306537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:43:12.306537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:12.306537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 480/942] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:43:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:12.307978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07b341-328d-4dbc-ada8-da4f86fc6cf0 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:12.307978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07b341-328d-4dbc-ada8-da4f86fc6cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:12.307978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07b341-328d-4dbc-ada8-da4f86fc6cf0 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:12.311748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 481/943] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:12.312569 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07b341-328d-4dbc-ada8-da4f86fc6cf0 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:12.336152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d91310-dee7-434a-bcd4-4797fdcdf576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:12.337096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d91310-dee7-434a-bcd4-4797fdcdf576 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:12.337589 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d91310-dee7-434a-bcd4-4797fdcdf576 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:12.338053 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d91310-dee7-434a-bcd4-4797fdcdf576 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:12.339120 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d91310-dee7-434a-bcd4-4797fdcdf576 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:12.339622 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d91310-dee7-434a-bcd4-4797fdcdf576 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:12.341243 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d91310-dee7-434a-bcd4-4797fdcdf576 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:12.361724 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 463/944] 10.222.0.17 () {56 vars in 1149 bytes} [Sat Sep 10 23:43:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:12.363410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fe8ca-0c0b-48d9-bc28-0a5b2573c70c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:12.363691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fe8ca-0c0b-48d9-bc28-0a5b2573c70c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:12.363959 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fe8ca-0c0b-48d9-bc28-0a5b2573c70c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:12.371210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:12.384440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c92fe8ca-0c0b-48d9-bc28-0a5b2573c70c None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:12.385679 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c92fe8ca-0c0b-48d9-bc28-0a5b2573c70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:12.438287 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07b341-328d-4dbc-ada8-da4f86fc6cf0 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:12.459921 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 464/945] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:43:12.460926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fe8ca-0c0b-48d9-bc28-0a5b2573c70c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:12.485787 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c1f6c-b558-44fd-bd3d-f257c612aba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:12.486787 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c1f6c-b558-44fd-bd3d-f257c612aba8 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:12.487243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c1f6c-b558-44fd-bd3d-f257c612aba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:12.487815 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c1f6c-b558-44fd-bd3d-f257c612aba8 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:12.488788 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c1f6c-b558-44fd-bd3d-f257c612aba8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:12.489281 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c1f6c-b558-44fd-bd3d-f257c612aba8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:12.491071 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c1f6c-b558-44fd-bd3d-f257c612aba8 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:12.510903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 482/946] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Sep 10 23:43:12 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:12.583890 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fe8ca-0c0b-48d9-bc28-0a5b2573c70c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:12.617996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:12.624288 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7494e-6c16-48f2-b593-4d9ad438d76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:12.624934 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7494e-6c16-48f2-b593-4d9ad438d76b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:12.625208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7494e-6c16-48f2-b593-4d9ad438d76b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:12.625479 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7494e-6c16-48f2-b593-4d9ad438d76b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:12.637750 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7494e-6c16-48f2-b593-4d9ad438d76b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:12.639330 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7494e-6c16-48f2-b593-4d9ad438d76b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:12.657922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-48b7494e-6c16-48f2-b593-4d9ad438d76b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:12.657922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:12.657922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:12.657922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:12.657922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:12.657922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:12.657922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:12.657922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:12.657922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:12.657922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:12.657922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:12.657922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:12.657922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:43:12.657922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:43:12.657922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:43:12.657922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:43:12.659004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:12.659004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:12.659004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:12.659004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:12.659004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:12.659004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:12.659004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:12.659004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:12.659004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:12.659004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:12.659004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:12.659004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:12.659004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:12.659004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:12.659004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:12.659004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:12.659004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:43:12.659885 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:43:12.659885 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:12.659885 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:12.659885 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 465/947] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:43:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:12.665455 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c1f6c-b558-44fd-bd3d-f257c612aba8 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:12.725273 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:12.773443 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624676cb-6792-48d9-a2af-6acb4066fc31 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:12.773749 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624676cb-6792-48d9-a2af-6acb4066fc31 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:12.773978 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624676cb-6792-48d9-a2af-6acb4066fc31 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:12.775361 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 466/948] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:12.776632 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624676cb-6792-48d9-a2af-6acb4066fc31 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:12.793943 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c1f6c-b558-44fd-bd3d-f257c612aba8 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:12.829882 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65abc06-9ce1-4fb0-b22a-fc8d1834ed77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:12.830891 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65abc06-9ce1-4fb0-b22a-fc8d1834ed77 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:12.831170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65abc06-9ce1-4fb0-b22a-fc8d1834ed77 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:12.831706 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65abc06-9ce1-4fb0-b22a-fc8d1834ed77 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:12.832343 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65abc06-9ce1-4fb0-b22a-fc8d1834ed77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:12.832640 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65abc06-9ce1-4fb0-b22a-fc8d1834ed77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:12.834057 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65abc06-9ce1-4fb0-b22a-fc8d1834ed77 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:12.847076 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 483/949] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:12.848485 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2873193-e857-40e0-b73a-5ef75775748e None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:12.848769 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2873193-e857-40e0-b73a-5ef75775748e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:12.849038 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2873193-e857-40e0-b73a-5ef75775748e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:12.870356 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-f2873193-e857-40e0-b73a-5ef75775748e None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:12.872520 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f2873193-e857-40e0-b73a-5ef75775748e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:12.891180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624676cb-6792-48d9-a2af-6acb4066fc31 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:12.926334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac6132-961a-43ed-8086-14fa187214df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:12.926978 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac6132-961a-43ed-8086-14fa187214df None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:12.927258 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac6132-961a-43ed-8086-14fa187214df None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:12.927519 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac6132-961a-43ed-8086-14fa187214df None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:12.928098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac6132-961a-43ed-8086-14fa187214df None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:12.929456 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac6132-961a-43ed-8086-14fa187214df None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:12.938607 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-5eac6132-961a-43ed-8086-14fa187214df None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 23:43:12.938607 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:12.938607 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 23:43:12.938607 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 23:43:12.938607 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 23:43:12.938607 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 23:43:12.938607 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 23:43:12.938607 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:12.938607 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 23:43:12.938607 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:12.938607 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:43:12.938607 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:12.938607 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:12.938607 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:12.938607 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:12.938607 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:12.938607 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:12.939697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:12.939697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:12.939697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:12.939697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:12.939697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:12.939697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:12.939697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 23:43:12.939697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 23:43:12.939697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 23:43:12.939697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 23:43:12.939697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:12.939697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:12.939697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:12.939697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:12.939697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:12.939697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:12.940568 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:12.940568 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:12.940568 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:12.940568 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:12.940568 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:12.940568 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:12.940568 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:12.940568 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:12.940568 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:12.940568 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:12.940568 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 23:43:12.940568 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:12.940568 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 23:43:12.940568 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:12.940568 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 467/950] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Sep 10 23:43:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:12.942043 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac6132-961a-43ed-8086-14fa187214df None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:12.959096 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 484/951] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:43:12.959848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2873193-e857-40e0-b73a-5ef75775748e None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:13.002831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:13.025185 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:13.084242 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac6132-961a-43ed-8086-14fa187214df None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:13.097433 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2873193-e857-40e0-b73a-5ef75775748e None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:13.120766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150f922-d409-44d5-8784-1ba5611ea9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:13.121769 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150f922-d409-44d5-8784-1ba5611ea9b3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:13.122057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150f922-d409-44d5-8784-1ba5611ea9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:13.122327 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150f922-d409-44d5-8784-1ba5611ea9b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:13.131248 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150f922-d409-44d5-8784-1ba5611ea9b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:13.133577 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150f922-d409-44d5-8784-1ba5611ea9b3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:13.141359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c2c9fc-abd7-4ca5-bb8e-af767926a91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:13.142166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2c9fc-abd7-4ca5-bb8e-af767926a91e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:13.142591 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2c9fc-abd7-4ca5-bb8e-af767926a91e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:13.143066 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2c9fc-abd7-4ca5-bb8e-af767926a91e None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:13.143838 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c2c9fc-abd7-4ca5-bb8e-af767926a91e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:13.144285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c2c9fc-abd7-4ca5-bb8e-af767926a91e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:13.145792 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c2c9fc-abd7-4ca5-bb8e-af767926a91e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:13.146142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-0150f922-d409-44d5-8784-1ba5611ea9b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:43:13.146142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:13.146142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:13.146142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:13.146142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:13.146142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:13.146142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:13.146142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:13.146142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:13.146142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:13.146142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:13.146142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:13.146142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:43:13.146142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:43:13.146142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:43:13.147210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:43:13.147210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:13.147210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:13.147210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:13.147210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:13.147210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:13.147210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:13.147210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:13.147210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:13.147210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:13.147210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:13.147210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:13.147210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:13.147210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:13.147210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:13.147210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:13.147210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:13.148545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:43:13.148545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:43:13.148545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:43:13.148545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:43:13.148545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:43:13.148545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:43:13.148545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:43:13.148545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:13.148545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 468/952] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:43:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:13.153463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150f922-d409-44d5-8784-1ba5611ea9b3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:13.156429 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 485/953] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Sep 10 23:43:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:13.161580 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c2c9fc-abd7-4ca5-bb8e-af767926a91e None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:13.259418 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150f922-d409-44d5-8784-1ba5611ea9b3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:13.269227 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:13.272125 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:13.279597 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c2c9fc-abd7-4ca5-bb8e-af767926a91e None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:13.298024 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7bfb22-a244-41ae-a788-988fe9b9a84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:13.299186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7bfb22-a244-41ae-a788-988fe9b9a84c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:13.299617 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7bfb22-a244-41ae-a788-988fe9b9a84c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:13.300057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7bfb22-a244-41ae-a788-988fe9b9a84c None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:13.300754 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7bfb22-a244-41ae-a788-988fe9b9a84c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:13.301063 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7bfb22-a244-41ae-a788-988fe9b9a84c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:13.303002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7bfb22-a244-41ae-a788-988fe9b9a84c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:13.317524 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce6a14-9aa1-4a3e-b93d-111a82b225ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:13.318921 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce6a14-9aa1-4a3e-b93d-111a82b225ce None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:13.318921 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce6a14-9aa1-4a3e-b93d-111a82b225ce None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:13.318921 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce6a14-9aa1-4a3e-b93d-111a82b225ce None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:13.319565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce6a14-9aa1-4a3e-b93d-111a82b225ce None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:13.319860 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce6a14-9aa1-4a3e-b93d-111a82b225ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:13.321234 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce6a14-9aa1-4a3e-b93d-111a82b225ce None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:13.324269 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 469/954] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:43:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:13.325100 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7bfb22-a244-41ae-a788-988fe9b9a84c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:13.330813 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 486/955] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Sep 10 23:43:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:13.330813 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce6a14-9aa1-4a3e-b93d-111a82b225ce None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:13.433484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7bfb22-a244-41ae-a788-988fe9b9a84c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:13.438178 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce6a14-9aa1-4a3e-b93d-111a82b225ce None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:13.473185 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae1e1b-f71f-429d-a2bd-e91b72e0e004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:13.473705 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34000982-7afa-4860-86f9-9c0250fdecce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:13.473797 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae1e1b-f71f-429d-a2bd-e91b72e0e004 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:13.474056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34000982-7afa-4860-86f9-9c0250fdecce None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:13.474165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae1e1b-f71f-429d-a2bd-e91b72e0e004 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:13.474314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34000982-7afa-4860-86f9-9c0250fdecce None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:13.474466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae1e1b-f71f-429d-a2bd-e91b72e0e004 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:13.474556 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34000982-7afa-4860-86f9-9c0250fdecce None admin] PATH_INFO: `/v3/projects/947277c4458645899287e6818947c076/users/b0a879085a194b0687e1cdcbafd48033/roles/24493ac356254954854aeec4ad5717d5` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:13.475062 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae1e1b-f71f-429d-a2bd-e91b72e0e004 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:13.476465 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae1e1b-f71f-429d-a2bd-e91b72e0e004 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:13.476980 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34000982-7afa-4860-86f9-9c0250fdecce None admin] RBAC: Authorizing `identity:create_grant(project_id=947277c4458645899287e6818947c076, user_id=b0a879085a194b0687e1cdcbafd48033, role_id=24493ac356254954854aeec4ad5717d5)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:13.478626 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34000982-7afa-4860-86f9-9c0250fdecce None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:13.486031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-20ae1e1b-f71f-429d-a2bd-e91b72e0e004 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 23:43:13.486031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:13.486031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 23:43:13.486031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 23:43:13.486031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 23:43:13.486031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 23:43:13.486031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 23:43:13.486031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:13.486031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 23:43:13.486031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:13.486031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:43:13.486031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:13.486031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:13.486031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:13.486031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:13.486031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:13.486031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:13.494092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:13.494092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:13.494092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:13.494092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:13.494092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:13.494092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:13.494092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 23:43:13.494092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 23:43:13.494092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 23:43:13.494092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 23:43:13.494092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:13.494092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:13.494092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:13.494092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:13.494092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:13.494092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:13.495176 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:13.495176 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:13.495176 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:13.495176 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:13.495176 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:13.495176 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:13.495176 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:13.495176 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:13.495176 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:13.495176 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:13.495176 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 23:43:13.495176 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:13.495176 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 23:43:13.495176 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:13.495176 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 470/956] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:43:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:13.495971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae1e1b-f71f-429d-a2bd-e91b72e0e004 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:13.518822 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:13.584539 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 487/957] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Sep 10 23:43:13 2022] PUT /identity/v3/projects/947277c4458645899287e6818947c076/users/b0a879085a194b0687e1cdcbafd48033/roles/24493ac356254954854aeec4ad5717d5 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:43:13.585659 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34000982-7afa-4860-86f9-9c0250fdecce None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:13.600762 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae1e1b-f71f-429d-a2bd-e91b72e0e004 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:13.608512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:13.654305 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95a879-5c9d-467f-8395-96c646e7f41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:13.654972 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95a879-5c9d-467f-8395-96c646e7f41f None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:13.655247 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95a879-5c9d-467f-8395-96c646e7f41f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:13.655516 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95a879-5c9d-467f-8395-96c646e7f41f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:13.656272 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95a879-5c9d-467f-8395-96c646e7f41f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:13.657771 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95a879-5c9d-467f-8395-96c646e7f41f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:13.710856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34000982-7afa-4860-86f9-9c0250fdecce None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:13.735065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 471/958] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Sep 10 23:43:13 2022] POST /identity/v3/endpoints => generated 349 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:43:13.769364 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16255b78-f75c-4002-909d-1cf32a7e27b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:13.769364 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16255b78-f75c-4002-909d-1cf32a7e27b3 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:13.769648 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16255b78-f75c-4002-909d-1cf32a7e27b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:13.769907 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16255b78-f75c-4002-909d-1cf32a7e27b3 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:13.770523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16255b78-f75c-4002-909d-1cf32a7e27b3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:13.770856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16255b78-f75c-4002-909d-1cf32a7e27b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:13.772680 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16255b78-f75c-4002-909d-1cf32a7e27b3 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:13.778474 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 488/959] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Sep 10 23:43:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:13.784707 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95a879-5c9d-467f-8395-96c646e7f41f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:13.890354 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:13.905002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95a879-5c9d-467f-8395-96c646e7f41f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:13.942976 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699c480-90fa-4fa2-8a6c-b85460a1f5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:13.943587 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699c480-90fa-4fa2-8a6c-b85460a1f5dc None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:13.943861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699c480-90fa-4fa2-8a6c-b85460a1f5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:13.944132 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699c480-90fa-4fa2-8a6c-b85460a1f5dc None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:13.944762 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699c480-90fa-4fa2-8a6c-b85460a1f5dc None admin] RBAC: Adding query filter params (type=compute) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:13.945061 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699c480-90fa-4fa2-8a6c-b85460a1f5dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:13.946424 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699c480-90fa-4fa2-8a6c-b85460a1f5dc None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:13.951874 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 472/960] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Sep 10 23:43:13 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:14.898573 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36adfc-3fdb-4cb1-893a-5ebe7a24ebc1 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:14.899148 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36adfc-3fdb-4cb1-893a-5ebe7a24ebc1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:14.899364 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36adfc-3fdb-4cb1-893a-5ebe7a24ebc1 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:14.902275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 489/961] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:14.906913 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a67c9c-b1a3-49ab-b564-84a09feaa2fc None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:14.907342 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a67c9c-b1a3-49ab-b564-84a09feaa2fc None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:14.907510 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a67c9c-b1a3-49ab-b564-84a09feaa2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:14.925743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-09a67c9c-b1a3-49ab-b564-84a09feaa2fc None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:14.926949 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-09a67c9c-b1a3-49ab-b564-84a09feaa2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:15.002775 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 473/962] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:43:15.273630 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36adfc-3fdb-4cb1-893a-5ebe7a24ebc1 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:15.369958 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:15.393634 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36adfc-3fdb-4cb1-893a-5ebe7a24ebc1 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:15.429964 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7edec3-78bf-4ca7-85bc-fa5adaea5c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:15.431044 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7edec3-78bf-4ca7-85bc-fa5adaea5c1f None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:15.431517 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7edec3-78bf-4ca7-85bc-fa5adaea5c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:15.431973 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7edec3-78bf-4ca7-85bc-fa5adaea5c1f None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:15.433173 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7edec3-78bf-4ca7-85bc-fa5adaea5c1f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:15.434804 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7edec3-78bf-4ca7-85bc-fa5adaea5c1f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:15.475963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.sql.core [None req-0d7edec3-78bf-4ca7-85bc-fa5adaea5c1f None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 10 23:43:15.475963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 10 23:43:15.475963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [parameters: {'user_id': '18ff573f676d48e4874f95499a98af0d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 10 23:43:15.475963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81573) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 10 23:43:15.483352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-0d7edec3-78bf-4ca7-85bc-fa5adaea5c1f None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 10 23:43:15.483352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:15.483352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 23:43:15.483352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 23:43:15.483352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 23:43:15.483352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 23:43:15.483352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 23:43:15.483352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 23:43:15.483352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 23:43:15.483352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application conn.query(q) Sep 10 23:43:15.483352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 23:43:15.483352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 23:43:15.483352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 23:43:15.483352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application result.read() Sep 10 23:43:15.483352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 23:43:15.484244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 23:43:15.484244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 23:43:15.484244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 23:43:15.484244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 23:43:15.484244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 23:43:15.484244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 23:43:15.484244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 23:43:15.484244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 10 23:43:15.484244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:15.484244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 10 23:43:15.484244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:15.484244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:15.484244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 10 23:43:15.484244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 10 23:43:15.484244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 10 23:43:15.484244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 10 23:43:15.484244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 23:43:15.484244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application next(self.gen) Sep 10 23:43:15.485200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 10 23:43:15.485200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application yield resource Sep 10 23:43:15.485200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 23:43:15.485200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application next(self.gen) Sep 10 23:43:15.485200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 10 23:43:15.485200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self.session.rollback() Sep 10 23:43:15.485200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 10 23:43:15.485200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self.force_reraise() Sep 10 23:43:15.485200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 10 23:43:15.485200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise self.value Sep 10 23:43:15.485200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 10 23:43:15.485200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 10 23:43:15.485200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 10 23:43:15.485200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application session.commit() Sep 10 23:43:15.485200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 10 23:43:15.485200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 10 23:43:15.485200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 10 23:43:15.486085 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._prepare_impl() Sep 10 23:43:15.486085 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 10 23:43:15.486085 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self.session.flush() Sep 10 23:43:15.486085 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 10 23:43:15.486085 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._flush(objects) Sep 10 23:43:15.486085 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 10 23:43:15.486085 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 10 23:43:15.486085 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 10 23:43:15.486085 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application compat.raise_( Sep 10 23:43:15.486085 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 23:43:15.486085 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception Sep 10 23:43:15.486085 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 10 23:43:15.486085 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application flush_context.execute() Sep 10 23:43:15.486085 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 10 23:43:15.486085 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rec.execute(self) Sep 10 23:43:15.486085 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 10 23:43:15.486085 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 10 23:43:15.487017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 10 23:43:15.487017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 10 23:43:15.487017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 10 23:43:15.487017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 10 23:43:15.487017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 10 23:43:15.487017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 10 23:43:15.487017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 10 23:43:15.487017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 10 23:43:15.487017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 10 23:43:15.487017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 10 23:43:15.487017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 10 23:43:15.487017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 10 23:43:15.487017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 10 23:43:15.487017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 10 23:43:15.487017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 23:43:15.487017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception Sep 10 23:43:15.487933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 23:43:15.487933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 23:43:15.487933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 23:43:15.487933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 23:43:15.487933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 23:43:15.487933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 23:43:15.487933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 23:43:15.487933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application conn.query(q) Sep 10 23:43:15.487933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 23:43:15.487933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 23:43:15.487933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 23:43:15.487933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application result.read() Sep 10 23:43:15.487933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 23:43:15.487933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 23:43:15.487933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 23:43:15.487933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 23:43:15.488755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 23:43:15.488755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 23:43:15.488755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 23:43:15.488755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 23:43:15.488755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 10 23:43:15.488755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 10 23:43:15.488755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [parameters: {'user_id': '18ff573f676d48e4874f95499a98af0d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 10 23:43:15.488755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 10 23:43:15.488755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:15.488755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:43:15.488755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:15.488755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:15.488755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:15.488755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:15.488755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:15.491523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:15.491523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:15.491523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:15.491523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:15.491523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:15.491523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:15.491523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:15.491523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 10 23:43:15.491523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 10 23:43:15.491523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:15.491523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:15.491523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:43:15.491523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:43:15.491523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:43:15.491523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:43:15.491523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 10 23:43:15.491523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 10 23:43:15.493637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 10 23:43:15.493637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 10 23:43:15.493637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 10 23:43:15.493637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 10 23:43:15.493637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 10 23:43:15.493637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:15.496685 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 490/963] 10.222.0.17 () {60 vars in 1131 bytes} [Sat Sep 10 23:43:15 2022] POST /identity/v3/users => generated 160 bytes in 223 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:15.502545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a67c9c-b1a3-49ab-b564-84a09feaa2fc None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:15.589093 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:15.607776 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a67c9c-b1a3-49ab-b564-84a09feaa2fc None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:15.646994 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8040845-2586-479a-8d4b-eb82235a883b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:15.647622 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8040845-2586-479a-8d4b-eb82235a883b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:15.647899 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8040845-2586-479a-8d4b-eb82235a883b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:15.648168 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8040845-2586-479a-8d4b-eb82235a883b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:15.656232 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8040845-2586-479a-8d4b-eb82235a883b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:15.657685 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8040845-2586-479a-8d4b-eb82235a883b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:15.670021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-c8040845-2586-479a-8d4b-eb82235a883b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 23:43:15.670021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:15.670021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:15.670021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:15.670021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:15.670021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:15.670021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:15.670021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:15.670021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:15.670021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:15.670021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:15.670021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:15.670021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:43:15.670021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:43:15.670021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:43:15.670021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:43:15.672213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:15.672213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:15.672213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:43:15.672213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:43:15.672213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:43:15.672213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:43:15.672213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:15.672213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:15.672213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:15.672213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:15.672213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:15.672213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:15.672213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:15.672213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:15.672213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:15.672213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:15.672213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:15.672213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:15.674338 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:15.674338 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:15.674338 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:43:15.674338 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:43:15.674338 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:43:15.674338 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:43:15.674338 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:43:15.674338 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:43:15.674338 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 23:43:15.674338 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:15.674338 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 474/964] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:43:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:15.677335 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7edec3-78bf-4ca7-85bc-fa5adaea5c1f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:15.795133 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7edec3-78bf-4ca7-85bc-fa5adaea5c1f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:15.830858 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77cb710-3ecc-4fcd-a9a3-f142f9d2204c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:15.831764 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cb710-3ecc-4fcd-a9a3-f142f9d2204c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:15.832212 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cb710-3ecc-4fcd-a9a3-f142f9d2204c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:15.832732 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cb710-3ecc-4fcd-a9a3-f142f9d2204c None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:15.833601 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77cb710-3ecc-4fcd-a9a3-f142f9d2204c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:15.834100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77cb710-3ecc-4fcd-a9a3-f142f9d2204c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:15.835913 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77cb710-3ecc-4fcd-a9a3-f142f9d2204c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:15.860908 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:15.875110 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 491/965] 10.222.0.17 () {56 vars in 1101 bytes} [Sat Sep 10 23:43:15 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:15.891004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04085b-4293-43d2-92b9-c897f240e36a None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:15.891279 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04085b-4293-43d2-92b9-c897f240e36a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:15.891558 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04085b-4293-43d2-92b9-c897f240e36a None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:15.893245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 475/966] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:15.899428 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950101a4-c396-40d2-ae50-4fb842a5246d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:15.899977 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950101a4-c396-40d2-ae50-4fb842a5246d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:15.900079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950101a4-c396-40d2-ae50-4fb842a5246d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:15.922250 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-950101a4-c396-40d2-ae50-4fb842a5246d None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:15.924379 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-950101a4-c396-40d2-ae50-4fb842a5246d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:16.009568 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 492/967] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:43:16.070245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c21ef-5cb4-4fc8-858b-935f1982563f None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:16.070776 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c21ef-5cb4-4fc8-858b-935f1982563f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:16.070854 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c21ef-5cb4-4fc8-858b-935f1982563f None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:16.072308 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 476/968] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:16.078419 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b529a0-553c-485b-a0f2-6120754b8b8c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:16.078754 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b529a0-553c-485b-a0f2-6120754b8b8c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:16.079009 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b529a0-553c-485b-a0f2-6120754b8b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:16.098908 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-28b529a0-553c-485b-a0f2-6120754b8b8c None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:16.100123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-28b529a0-553c-485b-a0f2-6120754b8b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:16.175492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 493/969] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:43:16.207931 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a8b1e-0f63-481f-bcc1-12b281eb5e8f None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:16.208250 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a8b1e-0f63-481f-bcc1-12b281eb5e8f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:16.208482 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a8b1e-0f63-481f-bcc1-12b281eb5e8f None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:16.209977 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 477/970] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:16.215846 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae72366-d838-4356-8305-346e5eb4ff5d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:16.216133 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae72366-d838-4356-8305-346e5eb4ff5d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:16.216407 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae72366-d838-4356-8305-346e5eb4ff5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:16.236015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-4ae72366-d838-4356-8305-346e5eb4ff5d None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:16.237192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4ae72366-d838-4356-8305-346e5eb4ff5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:16.285208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a8b1e-0f63-481f-bcc1-12b281eb5e8f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:16.291856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:16.310696 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 494/971] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:43:16.395090 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a8b1e-0f63-481f-bcc1-12b281eb5e8f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:16.428458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600bc2f0-e627-4e64-ab25-60da3f5f2e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:16.429004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600bc2f0-e627-4e64-ab25-60da3f5f2e95 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:16.429285 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600bc2f0-e627-4e64-ab25-60da3f5f2e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:16.429560 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600bc2f0-e627-4e64-ab25-60da3f5f2e95 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:16.440248 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600bc2f0-e627-4e64-ab25-60da3f5f2e95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:43:16.442160 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae72366-d838-4356-8305-346e5eb4ff5d None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:16.445092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-600bc2f0-e627-4e64-ab25-60da3f5f2e95 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 23:43:16.445092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:16.445092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:16.445092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:16.445092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:16.445092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:16.445092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:16.445092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:16.445092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:16.445092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:16.445092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:16.445092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:16.445092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:43:16.445092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:43:16.445092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:43:16.445092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:43:16.446023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:43:16.446023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:43:16.446023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:43:16.446023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:43:16.446023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:16.446023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:16.446023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:16.446023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:16.446023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:16.446023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:16.446023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:16.446023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:16.446023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:16.446023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:16.446023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:16.446023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:16.447389 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:16.447389 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:16.447389 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:16.447389 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:16.447389 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:43:16.447389 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:43:16.447389 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:43:16.447389 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:43:16.447389 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:43:16.447389 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:43:16.447389 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 23:43:16.447389 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:16.447389 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 478/972] 10.222.0.17 () {56 vars in 1120 bytes} [Sat Sep 10 23:43:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:43:16.455036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600bc2f0-e627-4e64-ab25-60da3f5f2e95 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:16.458241 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:16.554694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae72366-d838-4356-8305-346e5eb4ff5d None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:16.564620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600bc2f0-e627-4e64-ab25-60da3f5f2e95 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:16.591600 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe16621-78c3-47c5-bdd0-ba9c3822c559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:16.592230 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe16621-78c3-47c5-bdd0-ba9c3822c559 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:16.592507 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe16621-78c3-47c5-bdd0-ba9c3822c559 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:16.592783 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe16621-78c3-47c5-bdd0-ba9c3822c559 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:16.593366 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe16621-78c3-47c5-bdd0-ba9c3822c559 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:16.594757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe16621-78c3-47c5-bdd0-ba9c3822c559 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:16.600385 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458850a-4188-42ab-90ba-5b04ac736797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:16.600977 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458850a-4188-42ab-90ba-5b04ac736797 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:16.601257 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458850a-4188-42ab-90ba-5b04ac736797 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:16.601529 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458850a-4188-42ab-90ba-5b04ac736797 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:16.602184 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5458850a-4188-42ab-90ba-5b04ac736797 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:16.602490 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5458850a-4188-42ab-90ba-5b04ac736797 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:16.603503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-8fe16621-78c3-47c5-bdd0-ba9c3822c559 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 23:43:16.603503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:16.603503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 23:43:16.603503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 23:43:16.603503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 23:43:16.603503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 23:43:16.603503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 23:43:16.603503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:16.603503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 23:43:16.603503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:16.603503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:43:16.603503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:16.603503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:16.603503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:16.603503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:16.603503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:16.603503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:16.605393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:16.605393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:16.605393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:16.605393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:16.605393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:16.605393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:16.605393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 23:43:16.605393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 23:43:16.605393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 23:43:16.605393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 23:43:16.605393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:16.605393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:16.605393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:16.605393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:16.605393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:16.605393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:16.606223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:16.606223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:16.606223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:16.606223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:16.606223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:16.606223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:16.606223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:16.606223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:16.606223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:16.606223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:16.606223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 23:43:16.606223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:16.606223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 23:43:16.606223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:16.607002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5458850a-4188-42ab-90ba-5b04ac736797 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:16.607073 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 495/973] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Sep 10 23:43:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:43:16.607073 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe16621-78c3-47c5-bdd0-ba9c3822c559 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:16.618375 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 479/974] 10.222.0.17 () {56 vars in 1115 bytes} [Sat Sep 10 23:43:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:16.618375 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458850a-4188-42ab-90ba-5b04ac736797 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:16.657227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:16.676574 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:16.703126 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe16621-78c3-47c5-bdd0-ba9c3822c559 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:16.713304 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458850a-4188-42ab-90ba-5b04ac736797 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:16.736802 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735f289-1641-4d33-8264-365f4085d130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:16.737393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735f289-1641-4d33-8264-365f4085d130 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:16.737663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735f289-1641-4d33-8264-365f4085d130 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:16.737926 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735f289-1641-4d33-8264-365f4085d130 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:16.738562 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0735f289-1641-4d33-8264-365f4085d130 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:16.739936 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0735f289-1641-4d33-8264-365f4085d130 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:16.746750 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c519269-91f8-4a17-b404-25ad9c7087a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:16.747349 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c519269-91f8-4a17-b404-25ad9c7087a3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:16.747624 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c519269-91f8-4a17-b404-25ad9c7087a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:16.747891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c519269-91f8-4a17-b404-25ad9c7087a3 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:16.748515 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c519269-91f8-4a17-b404-25ad9c7087a3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:16.748817 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c519269-91f8-4a17-b404-25ad9c7087a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:16.750215 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c519269-91f8-4a17-b404-25ad9c7087a3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:16.754753 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 480/975] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Sep 10 23:43:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:16.755647 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c519269-91f8-4a17-b404-25ad9c7087a3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:16.803436 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 496/976] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Sep 10 23:43:16 2022] POST /identity/v3/services => generated 242 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:43:16.804258 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735f289-1641-4d33-8264-365f4085d130 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:16.872803 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c519269-91f8-4a17-b404-25ad9c7087a3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:16.910019 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:16.925942 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40fd2bf-4e85-46be-a0cb-9e2990849458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:16.926385 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fd2bf-4e85-46be-a0cb-9e2990849458 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:16.926677 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fd2bf-4e85-46be-a0cb-9e2990849458 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:16.926975 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fd2bf-4e85-46be-a0cb-9e2990849458 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:16.930935 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735f289-1641-4d33-8264-365f4085d130 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:16.939753 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40fd2bf-4e85-46be-a0cb-9e2990849458 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:16.942523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40fd2bf-4e85-46be-a0cb-9e2990849458 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:16.943038 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:16.959544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-f40fd2bf-4e85-46be-a0cb-9e2990849458 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:16.959544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:16.959544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:16.959544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:16.959544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:16.959544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:16.959544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:16.959544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:16.959544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:16.959544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:16.959544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:16.959544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:16.959544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:43:16.959544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:43:16.959544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:43:16.959544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:43:16.960526 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:16.960526 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:16.960526 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:16.960526 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:16.960526 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:16.960526 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:16.960526 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:16.960526 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:16.960526 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:16.960526 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:16.960526 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:16.960526 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:16.960526 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:16.960526 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:16.960526 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:16.960526 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:16.960526 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:43:16.961435 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:43:16.961435 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:16.961435 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:16.961435 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 481/977] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:43:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:16.966476 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40fd2bf-4e85-46be-a0cb-9e2990849458 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:16.973889 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcd250-d88e-45d0-8271-541888f9b3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:16.974507 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcd250-d88e-45d0-8271-541888f9b3c9 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:16.974856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcd250-d88e-45d0-8271-541888f9b3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:16.975126 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcd250-d88e-45d0-8271-541888f9b3c9 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:16.975763 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dcd250-d88e-45d0-8271-541888f9b3c9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:16.976064 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dcd250-d88e-45d0-8271-541888f9b3c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:16.977474 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dcd250-d88e-45d0-8271-541888f9b3c9 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:16.984253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 497/978] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Sep 10 23:43:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:16.989606 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcd250-d88e-45d0-8271-541888f9b3c9 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:17.083384 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40fd2bf-4e85-46be-a0cb-9e2990849458 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:17.102187 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcd250-d88e-45d0-8271-541888f9b3c9 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:17.119259 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c33cc-1949-4375-9cc0-a09ece9cc4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:17.120399 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c33cc-1949-4375-9cc0-a09ece9cc4e1 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:17.120881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c33cc-1949-4375-9cc0-a09ece9cc4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:17.121327 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c33cc-1949-4375-9cc0-a09ece9cc4e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:17.121972 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83c33cc-1949-4375-9cc0-a09ece9cc4e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:17.122389 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83c33cc-1949-4375-9cc0-a09ece9cc4e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:17.123804 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83c33cc-1949-4375-9cc0-a09ece9cc4e1 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:17.136563 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c66871-b15d-4a47-9933-d685e7c53c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:17.137160 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c66871-b15d-4a47-9933-d685e7c53c30 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:17.137433 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c66871-b15d-4a47-9933-d685e7c53c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:17.142221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c66871-b15d-4a47-9933-d685e7c53c30 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:17.142323 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 482/979] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:17.142419 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c66871-b15d-4a47-9933-d685e7c53c30 None admin] RBAC: Adding query filter params (interface=admin, service_id=10fea1fdca244d1887b4641c3389e730, region_id=RegionOne) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:17.142419 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c66871-b15d-4a47-9933-d685e7c53c30 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:17.142419 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c66871-b15d-4a47-9933-d685e7c53c30 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:17.142974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c33cc-1949-4375-9cc0-a09ece9cc4e1 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:17.146717 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 498/980] 10.222.0.17 () {56 vars in 1249 bytes} [Sat Sep 10 23:43:16 2022] GET /identity/v3/endpoints?service_id=10fea1fdca244d1887b4641c3389e730&interface=admin®ion_id=RegionOne => generated 193 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:17.152994 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c66871-b15d-4a47-9933-d685e7c53c30 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:17.179978 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:17.191976 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:17.247440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c33cc-1949-4375-9cc0-a09ece9cc4e1 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:17.266529 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c66871-b15d-4a47-9933-d685e7c53c30 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:17.281362 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b98dff-49ad-4e2b-b579-99f1792175d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:17.281983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b98dff-49ad-4e2b-b579-99f1792175d2 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:17.282266 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b98dff-49ad-4e2b-b579-99f1792175d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:17.282531 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b98dff-49ad-4e2b-b579-99f1792175d2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:17.289544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b98dff-49ad-4e2b-b579-99f1792175d2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:17.291002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b98dff-49ad-4e2b-b579-99f1792175d2 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:17.304427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-e1b98dff-49ad-4e2b-b579-99f1792175d2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 23:43:17.304427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:17.304427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:17.304427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:17.304427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:17.304427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:17.304427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:17.304427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:17.304427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:17.304427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:17.304427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:17.304427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:17.304427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:43:17.304427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:43:17.304427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:43:17.304427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:43:17.305400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:17.305400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:17.305400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:43:17.305400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:43:17.305400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:43:17.305400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:43:17.305400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:17.305400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:17.305400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:17.305400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:17.305400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:17.305400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:17.305400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:17.305400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:17.305400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:17.305400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:17.305400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:17.305400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:17.306348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:17.306348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:17.306348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:43:17.306348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:43:17.306348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:43:17.306348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:43:17.306348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:43:17.306348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:43:17.306348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 23:43:17.306348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:17.309241 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 483/981] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Sep 10 23:43:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:43:17.311040 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b98dff-49ad-4e2b-b579-99f1792175d2 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:17.313781 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10533bc0-317e-4689-a768-6499df2050ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:17.314394 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10533bc0-317e-4689-a768-6499df2050ce None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:17.315485 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10533bc0-317e-4689-a768-6499df2050ce None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:17.315485 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10533bc0-317e-4689-a768-6499df2050ce None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:17.315666 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10533bc0-317e-4689-a768-6499df2050ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:17.316886 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10533bc0-317e-4689-a768-6499df2050ce None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:17.328626 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 499/982] 10.222.0.17 () {56 vars in 1087 bytes} [Sat Sep 10 23:43:17 2022] GET /identity/v3/services => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 23:43:17.424414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:17.431000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b98dff-49ad-4e2b-b579-99f1792175d2 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:17.469015 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd38aaa-cefc-48dd-9227-d0ae1e16598f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:17.469627 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd38aaa-cefc-48dd-9227-d0ae1e16598f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:17.469902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd38aaa-cefc-48dd-9227-d0ae1e16598f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:17.470169 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd38aaa-cefc-48dd-9227-d0ae1e16598f None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:17.470949 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd38aaa-cefc-48dd-9227-d0ae1e16598f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:17.471251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd38aaa-cefc-48dd-9227-d0ae1e16598f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:17.472702 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd38aaa-cefc-48dd-9227-d0ae1e16598f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:17.493422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 484/983] 10.222.0.17 () {56 vars in 1149 bytes} [Sat Sep 10 23:43:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:17.500044 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10533bc0-317e-4689-a768-6499df2050ce None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:17.600238 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:17.603856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10533bc0-317e-4689-a768-6499df2050ce None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:17.637529 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288633f4-0ea5-4f0e-ba0c-40b9dd3378ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:17.638831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288633f4-0ea5-4f0e-ba0c-40b9dd3378ff None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:17.639092 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288633f4-0ea5-4f0e-ba0c-40b9dd3378ff None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:17.639372 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288633f4-0ea5-4f0e-ba0c-40b9dd3378ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:17.705889 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288633f4-0ea5-4f0e-ba0c-40b9dd3378ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:17.707351 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288633f4-0ea5-4f0e-ba0c-40b9dd3378ff None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:17.803177 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:17.808244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 500/984] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:43:17 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 10 23:43:17.813763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd38aaa-cefc-48dd-9227-d0ae1e16598f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:17.912381 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd38aaa-cefc-48dd-9227-d0ae1e16598f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:17.945628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d2422-78d5-4e04-8aea-d386b97f9a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:17.946235 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d2422-78d5-4e04-8aea-d386b97f9a25 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:17.946558 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d2422-78d5-4e04-8aea-d386b97f9a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:17.946860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d2422-78d5-4e04-8aea-d386b97f9a25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:17.956605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d2422-78d5-4e04-8aea-d386b97f9a25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:17.958040 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d2422-78d5-4e04-8aea-d386b97f9a25 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:17.973031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-647d2422-78d5-4e04-8aea-d386b97f9a25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:17.973031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:17.973031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:17.973031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:17.973031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:17.973031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:17.973031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:17.973031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:17.973031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:17.973031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:17.973031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:17.973031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:17.973031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:43:17.973031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:43:17.973031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:43:17.973031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:43:17.979708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:17.979708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:17.979708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:17.979708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:17.979708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:17.979708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:17.979708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:17.979708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:17.979708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:17.979708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:17.979708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:17.979708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:17.979708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:17.979708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:17.979708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:17.979708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:17.979708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:43:17.980622 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:43:17.980622 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:17.980622 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:17.980622 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 485/985] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:43:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:17.981942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288633f4-0ea5-4f0e-ba0c-40b9dd3378ff None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:18.082345 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288633f4-0ea5-4f0e-ba0c-40b9dd3378ff None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:18.116551 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90068bbc-a89b-4d12-abe1-770faa40bc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:18.117150 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90068bbc-a89b-4d12-abe1-770faa40bc68 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:18.117421 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90068bbc-a89b-4d12-abe1-770faa40bc68 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:18.117683 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90068bbc-a89b-4d12-abe1-770faa40bc68 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:18.118309 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90068bbc-a89b-4d12-abe1-770faa40bc68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:18.118603 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90068bbc-a89b-4d12-abe1-770faa40bc68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:18.120062 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90068bbc-a89b-4d12-abe1-770faa40bc68 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:18.136618 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 501/986] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:43:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:18.137854 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d2422-78d5-4e04-8aea-d386b97f9a25 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:18.147199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:18.239954 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d2422-78d5-4e04-8aea-d386b97f9a25 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:18.272761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cca126-fbc8-4502-99c8-f8b6d4ceb581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:18.273369 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cca126-fbc8-4502-99c8-f8b6d4ceb581 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:18.273641 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cca126-fbc8-4502-99c8-f8b6d4ceb581 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:18.273902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cca126-fbc8-4502-99c8-f8b6d4ceb581 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:18.281088 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cca126-fbc8-4502-99c8-f8b6d4ceb581 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:18.282506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cca126-fbc8-4502-99c8-f8b6d4ceb581 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:18.293606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-37cca126-fbc8-4502-99c8-f8b6d4ceb581 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:43:18.293606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:18.293606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:18.293606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:18.293606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:18.293606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:18.293606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:18.293606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:18.293606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:18.293606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:18.293606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:18.293606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:18.293606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:43:18.293606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:43:18.293606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:43:18.299080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:43:18.299080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:18.299080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:18.299080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:18.299080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:18.299080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:18.299080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:18.299080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:18.299080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:18.299080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:18.299080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:18.299080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:18.299080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:18.299080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:18.299080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:18.299080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:18.299080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:18.300002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:43:18.300002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:43:18.300002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:43:18.300002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:43:18.300002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:43:18.300002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:43:18.300002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:43:18.300002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:18.300002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 486/987] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:43:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:43:18.302848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90068bbc-a89b-4d12-abe1-770faa40bc68 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:18.368452 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:18.409826 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90068bbc-a89b-4d12-abe1-770faa40bc68 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:18.443105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f8e7e-1a3d-422c-91d1-6bb03ad3780f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:18.443716 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f8e7e-1a3d-422c-91d1-6bb03ad3780f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:18.443991 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f8e7e-1a3d-422c-91d1-6bb03ad3780f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:18.444258 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f8e7e-1a3d-422c-91d1-6bb03ad3780f None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:18.444946 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f8e7e-1a3d-422c-91d1-6bb03ad3780f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:18.445247 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f8e7e-1a3d-422c-91d1-6bb03ad3780f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:18.446652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f8e7e-1a3d-422c-91d1-6bb03ad3780f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:18.462757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 502/988] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:43:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:18.465577 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cca126-fbc8-4502-99c8-f8b6d4ceb581 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:18.532325 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:18.568781 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cca126-fbc8-4502-99c8-f8b6d4ceb581 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:18.602492 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791bec6-344a-4be3-aa26-66d4b67565da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:18.603114 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791bec6-344a-4be3-aa26-66d4b67565da None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:18.603392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791bec6-344a-4be3-aa26-66d4b67565da None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:18.603658 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791bec6-344a-4be3-aa26-66d4b67565da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:18.604933 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791bec6-344a-4be3-aa26-66d4b67565da None admin] RBAC: Adding query filter params (role.id=24493ac356254954854aeec4ad5717d5, scope.project.id=947277c4458645899287e6818947c076, user.id=b0a879085a194b0687e1cdcbafd48033) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:18.605233 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791bec6-344a-4be3-aa26-66d4b67565da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:18.606654 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791bec6-344a-4be3-aa26-66d4b67565da None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:18.612750 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 487/989] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:43:18 2022] GET /identity/v3/role_assignments?user.id=b0a879085a194b0687e1cdcbafd48033&scope.project.id=947277c4458645899287e6818947c076&role.id=24493ac356254954854aeec4ad5717d5 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:19.065525 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a676ff-1cac-4586-9af6-9dd559775b73 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:19.066091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a676ff-1cac-4586-9af6-9dd559775b73 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:19.066091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a676ff-1cac-4586-9af6-9dd559775b73 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:19.067711 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 503/990] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:19.073875 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5714ea6-c850-4dbe-958f-c293a232489c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:19.074252 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5714ea6-c850-4dbe-958f-c293a232489c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:19.074533 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5714ea6-c850-4dbe-958f-c293a232489c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:19.093265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a5714ea6-c850-4dbe-958f-c293a232489c None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:19.094457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a5714ea6-c850-4dbe-958f-c293a232489c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:19.164367 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 488/991] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:19 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:43:19.442936 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a676ff-1cac-4586-9af6-9dd559775b73 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:19.508647 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b00d93-a160-45dd-a989-eebc555d00c4 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:19.508923 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b00d93-a160-45dd-a989-eebc555d00c4 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:19.509195 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b00d93-a160-45dd-a989-eebc555d00c4 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:19.512163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 489/992] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:19.517905 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6306d-36f7-4737-a4ee-5b9d35f594cd None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:19.518239 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6306d-36f7-4737-a4ee-5b9d35f594cd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:19.518456 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6306d-36f7-4737-a4ee-5b9d35f594cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:19.540821 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1dc6306d-36f7-4737-a4ee-5b9d35f594cd None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:19.542045 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1dc6306d-36f7-4737-a4ee-5b9d35f594cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:19.555162 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a676ff-1cac-4586-9af6-9dd559775b73 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:19.577845 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:19.593858 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2a50f-19d3-44ae-8667-6d81324a5190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:19.594784 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2a50f-19d3-44ae-8667-6d81324a5190 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:19.595078 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2a50f-19d3-44ae-8667-6d81324a5190 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:19.595525 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2a50f-19d3-44ae-8667-6d81324a5190 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:19.596336 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d2a50f-19d3-44ae-8667-6d81324a5190 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:19.597706 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d2a50f-19d3-44ae-8667-6d81324a5190 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:19.601161 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:19.606825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-88d2a50f-19d3-44ae-8667-6d81324a5190 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 23:43:19.606825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:19.606825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 23:43:19.606825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 23:43:19.606825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 23:43:19.606825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 23:43:19.606825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 23:43:19.606825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:19.606825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 23:43:19.606825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:19.606825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:43:19.606825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:19.606825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:19.606825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:19.606825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:19.606825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:19.606825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:19.607811 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:19.607811 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:19.607811 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:19.607811 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:19.607811 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:19.607811 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:19.607811 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 23:43:19.607811 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 23:43:19.607811 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 23:43:19.607811 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 23:43:19.607811 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:19.607811 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:19.607811 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:19.607811 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:19.607811 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:19.607811 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:19.608649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:19.608649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:19.608649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:19.608649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:19.608649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:19.608649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:19.608649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:19.608649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:19.608649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:19.608649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:19.608649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 23:43:19.608649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:19.608649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 23:43:19.608649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:19.608649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 504/993] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:43:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:19.615335 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2a50f-19d3-44ae-8667-6d81324a5190 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:19.616550 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 490/994] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:19 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:43:19.717190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2a50f-19d3-44ae-8667-6d81324a5190 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:19.751703 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebdeff-8f89-4b72-a0a5-e17727aa7b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:19.752307 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebdeff-8f89-4b72-a0a5-e17727aa7b03 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:19.752580 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebdeff-8f89-4b72-a0a5-e17727aa7b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:19.752852 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebdeff-8f89-4b72-a0a5-e17727aa7b03 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:19.753467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ebdeff-8f89-4b72-a0a5-e17727aa7b03 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:19.753762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ebdeff-8f89-4b72-a0a5-e17727aa7b03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:19.755170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ebdeff-8f89-4b72-a0a5-e17727aa7b03 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:19.766694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 505/995] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Sep 10 23:43:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:19.769067 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6306d-36f7-4737-a4ee-5b9d35f594cd None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:19.873088 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6306d-36f7-4737-a4ee-5b9d35f594cd None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:19.891048 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebdeff-8f89-4b72-a0a5-e17727aa7b03 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:19.915951 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840cd36-693f-4761-9d14-47ca75efa56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:19.916688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840cd36-693f-4761-9d14-47ca75efa56e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:19.916998 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840cd36-693f-4761-9d14-47ca75efa56e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:19.917265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840cd36-693f-4761-9d14-47ca75efa56e None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:19.917893 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2840cd36-693f-4761-9d14-47ca75efa56e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:19.918187 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2840cd36-693f-4761-9d14-47ca75efa56e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:19.919606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2840cd36-693f-4761-9d14-47ca75efa56e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:19.924690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 491/996] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Sep 10 23:43:19 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:19.930820 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840cd36-693f-4761-9d14-47ca75efa56e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:19.953005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:19.988556 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:20.007015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebdeff-8f89-4b72-a0a5-e17727aa7b03 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:20.035564 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840cd36-693f-4761-9d14-47ca75efa56e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:20.043746 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66a051-b09d-457a-ac7e-cb5ae353b1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:20.044372 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66a051-b09d-457a-ac7e-cb5ae353b1c4 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:20.044767 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66a051-b09d-457a-ac7e-cb5ae353b1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:20.045038 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66a051-b09d-457a-ac7e-cb5ae353b1c4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:20.045616 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66a051-b09d-457a-ac7e-cb5ae353b1c4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:20.047155 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66a051-b09d-457a-ac7e-cb5ae353b1c4 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:20.061461 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-6a66a051-b09d-457a-ac7e-cb5ae353b1c4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 23:43:20.061461 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:20.061461 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 23:43:20.061461 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 23:43:20.061461 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 23:43:20.061461 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 23:43:20.061461 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 23:43:20.061461 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:20.061461 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 23:43:20.061461 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:20.061461 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:43:20.061461 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:20.061461 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:20.061461 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:20.061461 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:20.061461 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:20.061461 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:20.067257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:20.067257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:20.067257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:20.067257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:20.067257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:20.067257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:20.067257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 23:43:20.067257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 23:43:20.067257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 23:43:20.067257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 23:43:20.067257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:20.067257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:20.067257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:20.067257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:20.067257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:20.067257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:20.068275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:20.068275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:20.068275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:20.068275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:20.068275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:20.068275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:20.068275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:20.068275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:20.068275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:20.068275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:20.068275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 23:43:20.068275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:20.068275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 23:43:20.068275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:20.068275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 506/997] 10.222.0.17 () {56 vars in 1126 bytes} [Sat Sep 10 23:43:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:43:20.069249 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66a051-b09d-457a-ac7e-cb5ae353b1c4 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:20.080120 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8892e22-b6d6-48b7-bfe4-fa37e1a26abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:20.080947 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8892e22-b6d6-48b7-bfe4-fa37e1a26abc None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:20.081371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8892e22-b6d6-48b7-bfe4-fa37e1a26abc None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:20.081784 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8892e22-b6d6-48b7-bfe4-fa37e1a26abc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:20.082631 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8892e22-b6d6-48b7-bfe4-fa37e1a26abc None admin] RBAC: Adding query filter params (interface=public, service_id=3606f8c9e6c347b08a781d3134d675b6, region_id=RegionOne) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:20.083105 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8892e22-b6d6-48b7-bfe4-fa37e1a26abc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:20.084635 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8892e22-b6d6-48b7-bfe4-fa37e1a26abc None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:20.090044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 492/998] 10.222.0.17 () {56 vars in 1251 bytes} [Sat Sep 10 23:43:19 2022] GET /identity/v3/endpoints?service_id=3606f8c9e6c347b08a781d3134d675b6&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:20.095583 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8892e22-b6d6-48b7-bfe4-fa37e1a26abc None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:20.181301 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66a051-b09d-457a-ac7e-cb5ae353b1c4 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:20.195442 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:20.198814 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8892e22-b6d6-48b7-bfe4-fa37e1a26abc None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:20.217284 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:20.220218 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed6730-8af7-43b8-9c5b-791b3bfe632a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:20.221027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed6730-8af7-43b8-9c5b-791b3bfe632a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:20.221138 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed6730-8af7-43b8-9c5b-791b3bfe632a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:20.221347 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed6730-8af7-43b8-9c5b-791b3bfe632a None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:20.221980 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed6730-8af7-43b8-9c5b-791b3bfe632a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:20.222265 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed6730-8af7-43b8-9c5b-791b3bfe632a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:20.223713 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed6730-8af7-43b8-9c5b-791b3bfe632a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:20.230580 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 507/999] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Sep 10 23:43:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:20.236303 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed6730-8af7-43b8-9c5b-791b3bfe632a None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:20.236524 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ecd782-df59-40f3-8c2d-245aa1157327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:20.237129 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ecd782-df59-40f3-8c2d-245aa1157327 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:20.237406 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ecd782-df59-40f3-8c2d-245aa1157327 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:20.237677 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ecd782-df59-40f3-8c2d-245aa1157327 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:20.238278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ecd782-df59-40f3-8c2d-245aa1157327 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:20.239730 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ecd782-df59-40f3-8c2d-245aa1157327 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:20.254706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 493/1000] 10.222.0.17 () {56 vars in 1087 bytes} [Sat Sep 10 23:43:20 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 23:43:20.355075 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed6730-8af7-43b8-9c5b-791b3bfe632a None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:20.392658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd343bab-73c7-4d8c-94c3-fbabf7545244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:20.393279 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd343bab-73c7-4d8c-94c3-fbabf7545244 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:20.393551 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd343bab-73c7-4d8c-94c3-fbabf7545244 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:20.393816 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd343bab-73c7-4d8c-94c3-fbabf7545244 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:20.394454 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd343bab-73c7-4d8c-94c3-fbabf7545244 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:20.394781 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd343bab-73c7-4d8c-94c3-fbabf7545244 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:20.396154 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd343bab-73c7-4d8c-94c3-fbabf7545244 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:20.406493 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 508/1001] 10.222.0.17 () {56 vars in 1122 bytes} [Sat Sep 10 23:43:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:20.407785 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ecd782-df59-40f3-8c2d-245aa1157327 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:20.513586 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ecd782-df59-40f3-8c2d-245aa1157327 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:20.548147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8841a78-3680-4faa-a953-e07c3c40dc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:20.549116 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8841a78-3680-4faa-a953-e07c3c40dc58 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:20.549587 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8841a78-3680-4faa-a953-e07c3c40dc58 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:20.550056 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8841a78-3680-4faa-a953-e07c3c40dc58 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:20.550947 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8841a78-3680-4faa-a953-e07c3c40dc58 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:20.552499 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8841a78-3680-4faa-a953-e07c3c40dc58 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:20.643648 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 494/1002] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Sep 10 23:43:20 2022] POST /identity/v3/endpoints => generated 325 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 23:43:20.855542 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7526590-8e89-4e2f-ad87-e0f6a8fadccb None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:20.855839 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7526590-8e89-4e2f-ad87-e0f6a8fadccb None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:20.856018 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7526590-8e89-4e2f-ad87-e0f6a8fadccb None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:20.857482 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 509/1003] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:20.862935 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2b434-2101-4383-be43-23c66e6aa65a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:20.863217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2b434-2101-4383-be43-23c66e6aa65a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:20.863488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2b434-2101-4383-be43-23c66e6aa65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:20.882711 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-60d2b434-2101-4383-be43-23c66e6aa65a None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:20.883880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-60d2b434-2101-4383-be43-23c66e6aa65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:20.922180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:20.967186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 495/1004] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:20 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 23:43:21.238526 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7526590-8e89-4e2f-ad87-e0f6a8fadccb None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:21.283969 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:21.349239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7526590-8e89-4e2f-ad87-e0f6a8fadccb None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:21.384570 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41565f72-45a2-4465-8257-1cfaa6dca4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:21.385265 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41565f72-45a2-4465-8257-1cfaa6dca4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:21.385690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41565f72-45a2-4465-8257-1cfaa6dca4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:21.386180 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41565f72-45a2-4465-8257-1cfaa6dca4f8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:21.386799 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41565f72-45a2-4465-8257-1cfaa6dca4f8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:21.388163 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41565f72-45a2-4465-8257-1cfaa6dca4f8 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:21.397169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-41565f72-45a2-4465-8257-1cfaa6dca4f8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 23:43:21.397169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:21.397169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 23:43:21.397169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 23:43:21.397169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 23:43:21.397169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 23:43:21.397169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 23:43:21.397169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:21.397169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 23:43:21.397169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:21.397169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:43:21.397169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:21.397169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:21.397169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:21.397169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:21.397169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:21.397169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:21.398197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:21.398197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:21.398197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:21.398197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:21.398197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:21.398197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:21.398197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 23:43:21.398197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 23:43:21.398197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 23:43:21.398197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 23:43:21.398197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:21.398197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:21.398197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:21.398197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:21.398197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:21.398197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:21.399130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:21.399130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:21.399130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:21.399130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:21.399130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:21.399130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:21.399130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:21.399130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:21.399130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:21.399130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:21.399130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 23:43:21.399130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:21.399130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 23:43:21.399130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:21.399130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 510/1005] 10.222.0.17 () {56 vars in 1105 bytes} [Sat Sep 10 23:43:21 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:21.405672 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d2b434-2101-4383-be43-23c66e6aa65a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:21.509633 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d2b434-2101-4383-be43-23c66e6aa65a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:21.545636 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a7377-f99a-4893-8bd2-a89010eed464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:21.546242 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a7377-f99a-4893-8bd2-a89010eed464 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:21.546515 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a7377-f99a-4893-8bd2-a89010eed464 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:21.546814 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a7377-f99a-4893-8bd2-a89010eed464 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:21.547440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a7377-f99a-4893-8bd2-a89010eed464 None admin] RBAC: Adding query filter params (name=image) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:21.547737 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a7377-f99a-4893-8bd2-a89010eed464 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:21.549106 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a7377-f99a-4893-8bd2-a89010eed464 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:21.559667 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 496/1006] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:43:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:21.561342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41565f72-45a2-4465-8257-1cfaa6dca4f8 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:21.662278 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41565f72-45a2-4465-8257-1cfaa6dca4f8 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:21.664646 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:21.698482 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf773fd-821f-48e4-941c-f2d6e531eab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:21.699123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf773fd-821f-48e4-941c-f2d6e531eab4 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:21.699420 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf773fd-821f-48e4-941c-f2d6e531eab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:21.699782 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf773fd-821f-48e4-941c-f2d6e531eab4 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:21.700469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf773fd-821f-48e4-941c-f2d6e531eab4 None admin] RBAC: Adding query filter params (type=image) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:21.700772 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf773fd-821f-48e4-941c-f2d6e531eab4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:21.702303 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf773fd-821f-48e4-941c-f2d6e531eab4 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:21.707478 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 511/1007] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:43:21 2022] GET /identity/v3/services?type=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:22.403482 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa9ead-0775-4526-a775-1d7983826155 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:22.403828 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa9ead-0775-4526-a775-1d7983826155 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:22.404016 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa9ead-0775-4526-a775-1d7983826155 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:22.405489 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 497/1008] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:22.410564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654af3a-c9dd-4e31-ad5c-a70be8ef6f36 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:22.411027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654af3a-c9dd-4e31-ad5c-a70be8ef6f36 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:22.411213 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654af3a-c9dd-4e31-ad5c-a70be8ef6f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:22.429867 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-e654af3a-c9dd-4e31-ad5c-a70be8ef6f36 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:22.431069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e654af3a-c9dd-4e31-ad5c-a70be8ef6f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:22.498032 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 512/1009] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:22 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:43:22.763538 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa9ead-0775-4526-a775-1d7983826155 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:22.797614 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:22.868384 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa9ead-0775-4526-a775-1d7983826155 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:22.878497 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64927cb1-858d-4a93-81f6-25d60b35183f None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:22.878815 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64927cb1-858d-4a93-81f6-25d60b35183f None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:22.879088 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64927cb1-858d-4a93-81f6-25d60b35183f None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:22.880551 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 513/1010] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:22.885294 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284466f8-741f-4273-9bb9-52471d4c19ec None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:22.885637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284466f8-741f-4273-9bb9-52471d4c19ec None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:22.885917 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284466f8-741f-4273-9bb9-52471d4c19ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:22.903767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f021831-e312-4b96-8676-0fd63576fde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:22.904315 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-284466f8-741f-4273-9bb9-52471d4c19ec None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:22.904424 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f021831-e312-4b96-8676-0fd63576fde7 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:22.904639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f021831-e312-4b96-8676-0fd63576fde7 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:22.904905 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f021831-e312-4b96-8676-0fd63576fde7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:22.905478 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f021831-e312-4b96-8676-0fd63576fde7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:22.905583 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-284466f8-741f-4273-9bb9-52471d4c19ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:22.906844 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f021831-e312-4b96-8676-0fd63576fde7 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:22.915881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-7f021831-e312-4b96-8676-0fd63576fde7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 23:43:22.915881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:22.915881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 23:43:22.915881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 23:43:22.915881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 23:43:22.915881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 23:43:22.915881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 23:43:22.915881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:22.915881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 23:43:22.915881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:22.915881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:43:22.915881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:22.915881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:22.915881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:22.915881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:22.915881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:22.915881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:22.917467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:22.917467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:22.917467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:22.917467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:22.917467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:22.917467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:22.917467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 23:43:22.917467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 23:43:22.917467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 23:43:22.917467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 23:43:22.917467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:22.917467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:22.917467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:22.917467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:22.917467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:22.917467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:22.918378 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:22.918378 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:22.918378 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:22.918378 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:22.918378 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:22.918378 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:22.918378 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:22.918378 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:22.918378 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:22.918378 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:22.918378 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 23:43:22.918378 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:22.918378 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 23:43:22.918378 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:22.918378 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 498/1011] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:43:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:22.923527 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f021831-e312-4b96-8676-0fd63576fde7 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:22.986316 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 514/1012] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:22 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 23:43:23.065747 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:23.071506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f021831-e312-4b96-8676-0fd63576fde7 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:23.104321 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a47be0-9825-47f7-a91c-7de999158831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:23.104910 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a47be0-9825-47f7-a91c-7de999158831 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:23.105182 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a47be0-9825-47f7-a91c-7de999158831 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:23.105447 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a47be0-9825-47f7-a91c-7de999158831 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:23.106062 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a47be0-9825-47f7-a91c-7de999158831 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:23.106358 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a47be0-9825-47f7-a91c-7de999158831 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:23.107745 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a47be0-9825-47f7-a91c-7de999158831 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:23.113178 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 499/1013] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Sep 10 23:43:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:23.118378 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284466f8-741f-4273-9bb9-52471d4c19ec None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:23.146032 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:23.217118 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284466f8-741f-4273-9bb9-52471d4c19ec None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:23.250342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee2506-a3bd-42f6-8527-bad74d7a8406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:23.250968 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee2506-a3bd-42f6-8527-bad74d7a8406 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:23.251243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee2506-a3bd-42f6-8527-bad74d7a8406 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:23.251512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee2506-a3bd-42f6-8527-bad74d7a8406 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:23.252134 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee2506-a3bd-42f6-8527-bad74d7a8406 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:23.252434 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee2506-a3bd-42f6-8527-bad74d7a8406 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:23.253786 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee2506-a3bd-42f6-8527-bad74d7a8406 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:23.263554 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 515/1014] 10.222.0.17 () {56 vars in 1112 bytes} [Sat Sep 10 23:43:23 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:23.263706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a47be0-9825-47f7-a91c-7de999158831 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:23.264606 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee2506-a3bd-42f6-8527-bad74d7a8406 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:23.364242 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a47be0-9825-47f7-a91c-7de999158831 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:23.371267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:23.379626 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee2506-a3bd-42f6-8527-bad74d7a8406 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:23.400457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc49d69-7dca-40ef-ab85-9b2adeb0483c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:23.401799 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc49d69-7dca-40ef-ab85-9b2adeb0483c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:23.402216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc49d69-7dca-40ef-ab85-9b2adeb0483c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:23.402489 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc49d69-7dca-40ef-ab85-9b2adeb0483c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:23.403614 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc49d69-7dca-40ef-ab85-9b2adeb0483c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:23.405040 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc49d69-7dca-40ef-ab85-9b2adeb0483c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:23.407275 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 500/1015] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:43:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:23.412444 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc49d69-7dca-40ef-ab85-9b2adeb0483c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:23.415827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbad3eb-d3b0-4327-baf2-9a64d835a85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:23.417458 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbad3eb-d3b0-4327-baf2-9a64d835a85a None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:23.417745 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbad3eb-d3b0-4327-baf2-9a64d835a85a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:23.418021 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbad3eb-d3b0-4327-baf2-9a64d835a85a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:23.418720 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbad3eb-d3b0-4327-baf2-9a64d835a85a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:23.420115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbad3eb-d3b0-4327-baf2-9a64d835a85a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:23.445570 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:23.515442 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc49d69-7dca-40ef-ab85-9b2adeb0483c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:23.529933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 516/1016] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Sep 10 23:43:23 2022] POST /identity/v3/endpoints => generated 334 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:43:23.561007 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0c849-c4bf-4942-9a6b-3998df5cc0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:23.561624 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0c849-c4bf-4942-9a6b-3998df5cc0d4 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:23.561901 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0c849-c4bf-4942-9a6b-3998df5cc0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:23.562167 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0c849-c4bf-4942-9a6b-3998df5cc0d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:23.562890 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0c849-c4bf-4942-9a6b-3998df5cc0d4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:23.564368 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0c849-c4bf-4942-9a6b-3998df5cc0d4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:23.654868 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 501/1017] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Sep 10 23:43:23 2022] POST /identity/v3/projects => generated 311 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 23:43:23.936460 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde894b9-45c6-4a54-8cfa-a1bc609ad4d1 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:23.936837 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde894b9-45c6-4a54-8cfa-a1bc609ad4d1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:23.937014 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde894b9-45c6-4a54-8cfa-a1bc609ad4d1 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:23.938476 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 517/1018] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:23.943937 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e21fe-5910-4305-bb61-f37f244f2ad5 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:23.944435 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e21fe-5910-4305-bb61-f37f244f2ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:23.944622 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e21fe-5910-4305-bb61-f37f244f2ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:23.964860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-579e21fe-5910-4305-bb61-f37f244f2ad5 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:23.966055 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-579e21fe-5910-4305-bb61-f37f244f2ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:24.036549 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 502/1019] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:23 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:43:24.307833 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde894b9-45c6-4a54-8cfa-a1bc609ad4d1 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:24.415214 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde894b9-45c6-4a54-8cfa-a1bc609ad4d1 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:24.452936 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1261b5-b8a4-4e95-be6b-3dc9ee5333a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:24.453592 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1261b5-b8a4-4e95-be6b-3dc9ee5333a3 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:24.453884 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1261b5-b8a4-4e95-be6b-3dc9ee5333a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:24.454052 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1261b5-b8a4-4e95-be6b-3dc9ee5333a3 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:24.454721 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1261b5-b8a4-4e95-be6b-3dc9ee5333a3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:24.456105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1261b5-b8a4-4e95-be6b-3dc9ee5333a3 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:24.558501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 518/1020] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Sep 10 23:43:24 2022] POST /identity/v3/services => generated 242 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:43:25.852004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce0b11-8643-4267-899f-a3bc768cfe51 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:25.852550 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce0b11-8643-4267-899f-a3bc768cfe51 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:25.852550 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce0b11-8643-4267-899f-a3bc768cfe51 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:25.854715 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 503/1021] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:25.860149 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e8116-b5b0-4e29-a861-52b7d1b639e2 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:25.860587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e8116-b5b0-4e29-a861-52b7d1b639e2 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:25.860998 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e8116-b5b0-4e29-a861-52b7d1b639e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:25.862210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b848d6-3678-41ae-a877-b5dc46403ae4 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:25.862489 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b848d6-3678-41ae-a877-b5dc46403ae4 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:25.862786 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b848d6-3678-41ae-a877-b5dc46403ae4 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:25.864173 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 504/1022] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:43:25.868895 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516dd2f2-d9b3-46e9-a619-228c3f7ff013 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:25.869168 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516dd2f2-d9b3-46e9-a619-228c3f7ff013 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:25.869470 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516dd2f2-d9b3-46e9-a619-228c3f7ff013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:25.879221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b36e8116-b5b0-4e29-a861-52b7d1b639e2 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:25.880408 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b36e8116-b5b0-4e29-a861-52b7d1b639e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:25.881788 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:25.884109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:25.887444 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-516dd2f2-d9b3-46e9-a619-228c3f7ff013 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:25.888602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-516dd2f2-d9b3-46e9-a619-228c3f7ff013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:25.967002 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 519/1023] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:43:25.972032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 505/1024] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 23:43:26.252920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e8116-b5b0-4e29-a861-52b7d1b639e2 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:26.288594 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516dd2f2-d9b3-46e9-a619-228c3f7ff013 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:26.365651 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e8116-b5b0-4e29-a861-52b7d1b639e2 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:26.409600 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516dd2f2-d9b3-46e9-a619-228c3f7ff013 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:26.410601 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a3517a-0cda-4705-b256-56693cca4de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:26.411274 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a3517a-0cda-4705-b256-56693cca4de5 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:26.411546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a3517a-0cda-4705-b256-56693cca4de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:26.411810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a3517a-0cda-4705-b256-56693cca4de5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:26.412914 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a3517a-0cda-4705-b256-56693cca4de5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:26.414450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a3517a-0cda-4705-b256-56693cca4de5 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:26.417080 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 520/1025] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:43:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:26.425232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a3517a-0cda-4705-b256-56693cca4de5 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:26.427994 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:26.441229 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:26.446237 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e4a78-8109-46ff-8e17-e4e4379d1079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:26.446897 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e4a78-8109-46ff-8e17-e4e4379d1079 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:26.447172 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e4a78-8109-46ff-8e17-e4e4379d1079 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:26.447472 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e4a78-8109-46ff-8e17-e4e4379d1079 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:26.457832 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e4a78-8109-46ff-8e17-e4e4379d1079 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:43:26.462983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-727e4a78-8109-46ff-8e17-e4e4379d1079 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 23:43:26.462983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:26.462983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:26.462983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:26.462983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:26.462983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:26.462983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:26.462983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:26.462983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:26.462983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:26.462983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:26.462983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:26.462983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:43:26.462983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:43:26.462983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:43:26.462983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:43:26.464571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:43:26.464571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:43:26.464571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:43:26.464571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:43:26.464571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:26.464571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:26.464571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:26.464571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:26.464571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:26.464571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:26.464571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:26.464571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:26.464571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:26.464571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:26.464571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:26.464571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:26.465605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:26.465605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:26.465605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:26.465605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:26.465605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:43:26.465605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:43:26.465605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:43:26.465605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:43:26.465605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:43:26.465605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:43:26.465605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 23:43:26.465605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:26.465605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 506/1026] 10.222.0.17 () {56 vars in 1120 bytes} [Sat Sep 10 23:43:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:43:26.471094 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e4a78-8109-46ff-8e17-e4e4379d1079 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:26.539683 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a3517a-0cda-4705-b256-56693cca4de5 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:26.576244 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e4a78-8109-46ff-8e17-e4e4379d1079 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:26.578040 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b29329-a82b-439c-aaec-4efde444144d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:26.578643 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b29329-a82b-439c-aaec-4efde444144d None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:26.578940 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b29329-a82b-439c-aaec-4efde444144d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:26.579203 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b29329-a82b-439c-aaec-4efde444144d None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:26.579869 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b29329-a82b-439c-aaec-4efde444144d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:26.581279 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b29329-a82b-439c-aaec-4efde444144d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:26.612708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167fc2c-f9f9-41b1-a8c2-f0ebb5a5ab1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:26.613293 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167fc2c-f9f9-41b1-a8c2-f0ebb5a5ab1e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:26.613572 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167fc2c-f9f9-41b1-a8c2-f0ebb5a5ab1e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:26.613843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167fc2c-f9f9-41b1-a8c2-f0ebb5a5ab1e None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:26.614501 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c167fc2c-f9f9-41b1-a8c2-f0ebb5a5ab1e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:26.614838 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c167fc2c-f9f9-41b1-a8c2-f0ebb5a5ab1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:26.616214 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c167fc2c-f9f9-41b1-a8c2-f0ebb5a5ab1e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:26.625095 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 507/1027] 10.222.0.17 () {56 vars in 1115 bytes} [Sat Sep 10 23:43:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:26.631735 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167fc2c-f9f9-41b1-a8c2-f0ebb5a5ab1e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:26.678909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 521/1028] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Sep 10 23:43:26 2022] POST /identity/v3/users => generated 285 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 23:43:26.709450 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:26.752034 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b410f-66cc-41ea-b556-02eeb385fd20 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:26.752310 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b410f-66cc-41ea-b556-02eeb385fd20 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:26.752584 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b410f-66cc-41ea-b556-02eeb385fd20 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:26.753947 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 522/1029] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:43:26.759637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf575f53-da68-46d9-972b-c82c7a40b321 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:26.759979 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf575f53-da68-46d9-972b-c82c7a40b321 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:26.760437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf575f53-da68-46d9-972b-c82c7a40b321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:26.762911 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167fc2c-f9f9-41b1-a8c2-f0ebb5a5ab1e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:26.779278 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-bf575f53-da68-46d9-972b-c82c7a40b321 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:26.780549 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-bf575f53-da68-46d9-972b-c82c7a40b321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:26.800813 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6bfee-37b7-4652-80b9-28547b6bbf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:26.801367 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6bfee-37b7-4652-80b9-28547b6bbf62 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:26.801646 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6bfee-37b7-4652-80b9-28547b6bbf62 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:26.801918 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6bfee-37b7-4652-80b9-28547b6bbf62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:26.814746 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6bfee-37b7-4652-80b9-28547b6bbf62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:26.816374 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6bfee-37b7-4652-80b9-28547b6bbf62 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:26.833523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-eeb6bfee-37b7-4652-80b9-28547b6bbf62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:26.833523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:26.833523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:26.833523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:26.833523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:26.833523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:26.833523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:26.833523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:26.833523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:26.833523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:26.833523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:26.833523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:26.833523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:43:26.833523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:43:26.833523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:43:26.833523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:43:26.834623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:26.834623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:26.834623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:26.834623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:26.834623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:26.834623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:26.834623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:26.834623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:26.834623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:26.834623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:26.834623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:26.834623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:26.834623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:26.834623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:26.834623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:26.834623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:26.834623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:43:26.836083 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:43:26.836083 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:26.836083 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:26.836083 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 508/1030] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:43:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:43:26.840902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6bfee-37b7-4652-80b9-28547b6bbf62 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:26.842691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:26.859305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 523/1031] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 23:43:26.956702 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6bfee-37b7-4652-80b9-28547b6bbf62 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:26.965393 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:26.993125 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae334a-e46d-4845-b26c-fb577434d48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:26.993768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae334a-e46d-4845-b26c-fb577434d48f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:26.994044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae334a-e46d-4845-b26c-fb577434d48f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:26.994362 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae334a-e46d-4845-b26c-fb577434d48f None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:26.995120 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae334a-e46d-4845-b26c-fb577434d48f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:26.995450 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae334a-e46d-4845-b26c-fb577434d48f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:26.996809 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae334a-e46d-4845-b26c-fb577434d48f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:27.007998 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 509/1032] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:27.014197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf575f53-da68-46d9-972b-c82c7a40b321 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:27.116379 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf575f53-da68-46d9-972b-c82c7a40b321 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:27.137006 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae334a-e46d-4845-b26c-fb577434d48f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:27.152454 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbe8a3-0bd7-431b-ace0-508b147013d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:27.153156 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbe8a3-0bd7-431b-ace0-508b147013d0 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:27.153367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbe8a3-0bd7-431b-ace0-508b147013d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:27.153616 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbe8a3-0bd7-431b-ace0-508b147013d0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:27.161594 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dbe8a3-0bd7-431b-ace0-508b147013d0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:27.163074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dbe8a3-0bd7-431b-ace0-508b147013d0 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:27.175358 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-d7dbe8a3-0bd7-431b-ace0-508b147013d0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 23:43:27.175358 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:27.175358 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:27.175358 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:27.175358 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:27.175358 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:27.175358 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:27.175358 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:27.175358 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:27.175358 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:27.175358 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:27.175358 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:27.175358 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:43:27.175358 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:43:27.175358 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:43:27.175358 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:43:27.176329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:27.176329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:27.176329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:43:27.176329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:43:27.176329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:43:27.176329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:43:27.176329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:27.176329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:27.176329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:27.176329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:27.176329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:27.176329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:27.176329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:27.176329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:27.176329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:27.176329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:27.176329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:27.176329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:27.177323 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:27.177323 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:27.177323 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:43:27.177323 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:43:27.177323 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:43:27.177323 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:43:27.177323 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:43:27.177323 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:43:27.177323 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 23:43:27.177323 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:27.177323 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 524/1033] 10.222.0.17 () {56 vars in 1093 bytes} [Sat Sep 10 23:43:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:27.182076 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbe8a3-0bd7-431b-ace0-508b147013d0 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:27.230742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:27.251876 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae334a-e46d-4845-b26c-fb577434d48f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:27.286146 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbe8a3-0bd7-431b-ace0-508b147013d0 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:27.287121 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb16340-0e88-4c69-8602-3b91208b27f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:27.287750 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb16340-0e88-4c69-8602-3b91208b27f3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:27.288028 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb16340-0e88-4c69-8602-3b91208b27f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:27.288306 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb16340-0e88-4c69-8602-3b91208b27f3 None admin] PATH_INFO: `/v3/services/image` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:27.288893 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb16340-0e88-4c69-8602-3b91208b27f3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:27.290257 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb16340-0e88-4c69-8602-3b91208b27f3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:27.300186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-aeb16340-0e88-4c69-8602-3b91208b27f3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 23:43:27.300186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:27.300186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 23:43:27.300186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 23:43:27.300186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 23:43:27.300186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 23:43:27.300186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 23:43:27.300186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:27.300186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 23:43:27.300186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:27.300186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:43:27.300186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:27.300186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:27.300186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:27.300186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:27.300186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:27.300186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:27.301393 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:27.301393 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:27.301393 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:27.301393 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:27.301393 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:27.301393 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:27.301393 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 23:43:27.301393 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 23:43:27.301393 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 23:43:27.301393 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 23:43:27.301393 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:27.301393 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:27.301393 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:27.301393 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:27.301393 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:27.301393 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:27.302394 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:27.302394 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:27.302394 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:27.302394 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:27.302394 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:27.302394 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:27.302394 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:27.302394 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:27.302394 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:27.302394 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:27.302394 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 23:43:27.302394 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:27.302394 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 23:43:27.302394 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:27.306330 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 510/1034] 10.222.0.17 () {56 vars in 1105 bytes} [Sat Sep 10 23:43:27 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:43:27.307508 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb16340-0e88-4c69-8602-3b91208b27f3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:27.331586 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7df56c-4570-45ff-a792-0e9ba5c97a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:27.332188 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7df56c-4570-45ff-a792-0e9ba5c97a99 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:27.332465 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7df56c-4570-45ff-a792-0e9ba5c97a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:27.332742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7df56c-4570-45ff-a792-0e9ba5c97a99 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:27.333568 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7df56c-4570-45ff-a792-0e9ba5c97a99 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:27.333874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7df56c-4570-45ff-a792-0e9ba5c97a99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:27.335482 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7df56c-4570-45ff-a792-0e9ba5c97a99 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:27.342494 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:27.356880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 525/1035] 10.222.0.17 () {56 vars in 1133 bytes} [Sat Sep 10 23:43:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:27.362386 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7df56c-4570-45ff-a792-0e9ba5c97a99 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:27.414756 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb16340-0e88-4c69-8602-3b91208b27f3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:27.452443 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9be27-4c92-4711-86fe-b69e98a2d105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:27.453044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9be27-4c92-4711-86fe-b69e98a2d105 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:27.453325 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9be27-4c92-4711-86fe-b69e98a2d105 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:27.453598 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9be27-4c92-4711-86fe-b69e98a2d105 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:27.454236 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9be27-4c92-4711-86fe-b69e98a2d105 None admin] RBAC: Adding query filter params (name=image) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:27.454539 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9be27-4c92-4711-86fe-b69e98a2d105 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:27.456079 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9be27-4c92-4711-86fe-b69e98a2d105 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:27.461175 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 511/1036] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:43:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:27.466787 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9be27-4c92-4711-86fe-b69e98a2d105 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:27.469656 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7df56c-4570-45ff-a792-0e9ba5c97a99 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:27.477987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:27.507403 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49a0df-7e47-4695-a25e-bb28f41dd283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:27.508421 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49a0df-7e47-4695-a25e-bb28f41dd283 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:27.508695 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49a0df-7e47-4695-a25e-bb28f41dd283 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:27.508960 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49a0df-7e47-4695-a25e-bb28f41dd283 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:27.565927 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:27.574236 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9be27-4c92-4711-86fe-b69e98a2d105 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:27.585447 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e49a0df-7e47-4695-a25e-bb28f41dd283 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:27.586919 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e49a0df-7e47-4695-a25e-bb28f41dd283 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:27.610976 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa78b48-a4a4-4c35-b3f6-c62800bff6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:27.611605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa78b48-a4a4-4c35-b3f6-c62800bff6cf None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:27.611881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa78b48-a4a4-4c35-b3f6-c62800bff6cf None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:27.612153 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa78b48-a4a4-4c35-b3f6-c62800bff6cf None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:27.612780 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa78b48-a4a4-4c35-b3f6-c62800bff6cf None admin] RBAC: Adding query filter params (type=image) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:27.613080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa78b48-a4a4-4c35-b3f6-c62800bff6cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:27.614439 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa78b48-a4a4-4c35-b3f6-c62800bff6cf None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:27.619919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 512/1037] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:43:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:27.626435 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa78b48-a4a4-4c35-b3f6-c62800bff6cf None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:27.703849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 526/1038] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:43:27 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 10 23:43:27.709611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49a0df-7e47-4695-a25e-bb28f41dd283 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:27.732995 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:27.755344 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa78b48-a4a4-4c35-b3f6-c62800bff6cf None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:27.790901 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f30e3b-45bf-45df-9c30-8f0b82a974d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:27.791704 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f30e3b-45bf-45df-9c30-8f0b82a974d0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:27.792096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f30e3b-45bf-45df-9c30-8f0b82a974d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:27.792371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f30e3b-45bf-45df-9c30-8f0b82a974d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:27.793066 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f30e3b-45bf-45df-9c30-8f0b82a974d0 None admin] RBAC: Adding query filter params (interface=public, service_id=65ce1328ffdd420697091a66fac64f0b, region_id=RegionOne) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:27.793372 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f30e3b-45bf-45df-9c30-8f0b82a974d0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:27.794755 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f30e3b-45bf-45df-9c30-8f0b82a974d0 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:27.800288 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 513/1039] 10.222.0.17 () {56 vars in 1251 bytes} [Sat Sep 10 23:43:27 2022] GET /identity/v3/endpoints?service_id=65ce1328ffdd420697091a66fac64f0b&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:27.805693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f30e3b-45bf-45df-9c30-8f0b82a974d0 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:27.831860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:27.832082 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49a0df-7e47-4695-a25e-bb28f41dd283 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:27.868680 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c6dcf-52ac-4fa8-88de-0ca34e51b177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:27.869838 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c6dcf-52ac-4fa8-88de-0ca34e51b177 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:27.870266 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c6dcf-52ac-4fa8-88de-0ca34e51b177 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:27.870758 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c6dcf-52ac-4fa8-88de-0ca34e51b177 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:27.880607 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c6dcf-52ac-4fa8-88de-0ca34e51b177 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:27.882039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c6dcf-52ac-4fa8-88de-0ca34e51b177 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:27.897773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-e04c6dcf-52ac-4fa8-88de-0ca34e51b177 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:27.897773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:27.897773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:27.897773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:27.897773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:27.897773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:27.897773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:27.897773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:27.897773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:27.897773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:27.897773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:27.897773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:27.897773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:43:27.897773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:43:27.897773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:43:27.897773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:43:27.899046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:27.899046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:27.899046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:27.899046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:27.899046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:27.899046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:27.899046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:27.899046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:27.899046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:27.899046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:27.899046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:27.899046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:27.899046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:27.899046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:27.899046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:27.899046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:27.899046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:43:27.899963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:43:27.899963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:27.899963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:27.899963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 527/1040] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:43:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:43:27.904720 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c6dcf-52ac-4fa8-88de-0ca34e51b177 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:27.920175 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f30e3b-45bf-45df-9c30-8f0b82a974d0 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:27.958979 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3eb7c-d2d5-4504-98f7-67012b211291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:27.959606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3eb7c-d2d5-4504-98f7-67012b211291 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:27.960280 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3eb7c-d2d5-4504-98f7-67012b211291 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:27.960280 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3eb7c-d2d5-4504-98f7-67012b211291 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:27.960603 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d3eb7c-d2d5-4504-98f7-67012b211291 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:27.961989 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d3eb7c-d2d5-4504-98f7-67012b211291 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:27.966804 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:27.969416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 514/1041] 10.222.0.17 () {56 vars in 1087 bytes} [Sat Sep 10 23:43:27 2022] GET /identity/v3/services => generated 1957 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 23:43:28.016383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c6dcf-52ac-4fa8-88de-0ca34e51b177 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:28.064914 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3fead-b4ce-4a7e-823b-80c338503507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:28.065534 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3fead-b4ce-4a7e-823b-80c338503507 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:28.065810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3fead-b4ce-4a7e-823b-80c338503507 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:28.066080 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3fead-b4ce-4a7e-823b-80c338503507 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:28.066763 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e3fead-b4ce-4a7e-823b-80c338503507 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:28.067065 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e3fead-b4ce-4a7e-823b-80c338503507 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:28.068487 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e3fead-b4ce-4a7e-823b-80c338503507 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:28.082352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 528/1042] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:43:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:28.088611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3eb7c-d2d5-4504-98f7-67012b211291 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:28.181209 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:28.196969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3eb7c-d2d5-4504-98f7-67012b211291 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:28.232224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800a169-53ac-4ead-a3b2-0966b15fcf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:28.233102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800a169-53ac-4ead-a3b2-0966b15fcf01 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:28.233553 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800a169-53ac-4ead-a3b2-0966b15fcf01 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:28.233997 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800a169-53ac-4ead-a3b2-0966b15fcf01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:28.240798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800a169-53ac-4ead-a3b2-0966b15fcf01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:28.242437 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800a169-53ac-4ead-a3b2-0966b15fcf01 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:28.254196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-9800a169-53ac-4ead-a3b2-0966b15fcf01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:43:28.254196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:28.254196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:28.254196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:28.254196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:28.254196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:28.254196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:28.254196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:28.254196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:28.254196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:28.254196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:28.254196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:28.254196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:43:28.254196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:43:28.254196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:43:28.255112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:43:28.255112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:28.255112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:28.255112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:28.255112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:28.255112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:28.255112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:28.255112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:28.255112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:28.255112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:28.255112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:28.255112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:28.255112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:28.255112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:28.255112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:28.255112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:28.255112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:28.256000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:43:28.256000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:43:28.256000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:43:28.256000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:43:28.256000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:43:28.256000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:43:28.256000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:43:28.256000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:28.258726 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 515/1043] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:43:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:28.262429 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3fead-b4ce-4a7e-823b-80c338503507 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:28.372006 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3fead-b4ce-4a7e-823b-80c338503507 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:28.389474 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:28.406213 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e537a3-21b5-4d15-b868-52e07805f3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:28.406944 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e537a3-21b5-4d15-b868-52e07805f3bf None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:28.407226 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e537a3-21b5-4d15-b868-52e07805f3bf None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:28.407493 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e537a3-21b5-4d15-b868-52e07805f3bf None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:28.408178 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e537a3-21b5-4d15-b868-52e07805f3bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:28.408476 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e537a3-21b5-4d15-b868-52e07805f3bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:28.409947 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e537a3-21b5-4d15-b868-52e07805f3bf None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:28.427625 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 529/1044] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:43:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:28.429291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800a169-53ac-4ead-a3b2-0966b15fcf01 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:28.532205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800a169-53ac-4ead-a3b2-0966b15fcf01 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:28.566685 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c2781-dc37-489a-8103-cfd5fd950f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:28.567301 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c2781-dc37-489a-8103-cfd5fd950f1d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:28.567577 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c2781-dc37-489a-8103-cfd5fd950f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:28.567843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c2781-dc37-489a-8103-cfd5fd950f1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:28.568558 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c2781-dc37-489a-8103-cfd5fd950f1d None admin] RBAC: Adding query filter params (role.id=24493ac356254954854aeec4ad5717d5, scope.project.id=947277c4458645899287e6818947c076, user.id=636bfafb693044d283530063047fc0c2) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:28.568860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c2781-dc37-489a-8103-cfd5fd950f1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:28.570277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c2781-dc37-489a-8103-cfd5fd950f1d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:28.582927 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 516/1045] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:43:28 2022] GET /identity/v3/role_assignments?user.id=636bfafb693044d283530063047fc0c2&scope.project.id=947277c4458645899287e6818947c076&role.id=24493ac356254954854aeec4ad5717d5 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:28.970196 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b8e82-1799-444c-98fb-220e34518009 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:28.975622 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b8e82-1799-444c-98fb-220e34518009 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:28.975622 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b8e82-1799-444c-98fb-220e34518009 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:28.975622 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 530/1046] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:28.981404 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb3d15-f32f-468f-a6ea-25bf28d2a980 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:28.981861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb3d15-f32f-468f-a6ea-25bf28d2a980 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:28.981955 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb3d15-f32f-468f-a6ea-25bf28d2a980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:29.001389 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a0eb3d15-f32f-468f-a6ea-25bf28d2a980 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:29.002565 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a0eb3d15-f32f-468f-a6ea-25bf28d2a980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:29.005414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:29.079591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 517/1047] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:28 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:43:29.348444 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b8e82-1799-444c-98fb-220e34518009 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:29.457815 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b8e82-1799-444c-98fb-220e34518009 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:29.490613 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7f02f-02a3-41d1-9b2a-2c10235a6ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:29.491306 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7f02f-02a3-41d1-9b2a-2c10235a6ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:29.491587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7f02f-02a3-41d1-9b2a-2c10235a6ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:29.491850 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7f02f-02a3-41d1-9b2a-2c10235a6ec5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:29.503061 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d7f02f-02a3-41d1-9b2a-2c10235a6ec5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:43:29.507762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-b0d7f02f-02a3-41d1-9b2a-2c10235a6ec5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:43:29.507762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:29.507762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:29.507762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:29.507762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:29.507762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:29.507762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:29.507762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:29.507762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:29.507762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:29.507762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:29.507762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:29.507762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:43:29.507762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:43:29.507762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:43:29.507762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:43:29.508670 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:43:29.508670 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:43:29.508670 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:43:29.508670 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:43:29.508670 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:29.508670 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:29.508670 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:29.508670 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:29.508670 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:29.508670 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:29.508670 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:29.508670 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:29.508670 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:29.508670 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:29.508670 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:29.508670 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:29.509653 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:29.509653 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:29.509653 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:29.509653 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:29.509653 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:43:29.509653 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:43:29.509653 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:43:29.509653 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:43:29.509653 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:43:29.509653 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:43:29.509653 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:43:29.509653 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:29.509653 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 531/1048] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:43:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:29.514836 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb3d15-f32f-468f-a6ea-25bf28d2a980 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:29.613889 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb3d15-f32f-468f-a6ea-25bf28d2a980 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:29.647866 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ffedd-a4db-437a-98df-5c6a45595b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:29.648458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ffedd-a4db-437a-98df-5c6a45595b8e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:29.648728 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ffedd-a4db-437a-98df-5c6a45595b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:29.648992 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ffedd-a4db-437a-98df-5c6a45595b8e None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:29.649644 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ffedd-a4db-437a-98df-5c6a45595b8e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:29.649938 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ffedd-a4db-437a-98df-5c6a45595b8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:29.651930 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ffedd-a4db-437a-98df-5c6a45595b8e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:29.664853 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 518/1049] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:43:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:29.667460 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7f02f-02a3-41d1-9b2a-2c10235a6ec5 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:29.708608 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:29.769271 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7f02f-02a3-41d1-9b2a-2c10235a6ec5 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:29.805104 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f9a08-0316-49e3-b19c-7fa948f37b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:29.805731 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f9a08-0316-49e3-b19c-7fa948f37b52 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:29.806006 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f9a08-0316-49e3-b19c-7fa948f37b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:29.806273 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f9a08-0316-49e3-b19c-7fa948f37b52 None admin] PATH_INFO: `/v3/users/3491509777d44f7eb4e99fe21bf50133` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:29.820346 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f9a08-0316-49e3-b19c-7fa948f37b52 None admin] RBAC: Authorizing `identity:get_user(user_id=3491509777d44f7eb4e99fe21bf50133)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:29.821779 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f9a08-0316-49e3-b19c-7fa948f37b52 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:29.824190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 532/1050] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:43:29 2022] GET /identity/v3/users/3491509777d44f7eb4e99fe21bf50133 => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:29.829693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ffedd-a4db-437a-98df-5c6a45595b8e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:29.832552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:29.928875 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ffedd-a4db-437a-98df-5c6a45595b8e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:29.962309 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc61ea-84c9-4d89-88a7-90bb15fa5ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:29.962929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc61ea-84c9-4d89-88a7-90bb15fa5ccb None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:29.964262 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc61ea-84c9-4d89-88a7-90bb15fa5ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:29.964262 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc61ea-84c9-4d89-88a7-90bb15fa5ccb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:30.036116 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc61ea-84c9-4d89-88a7-90bb15fa5ccb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:30.037567 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc61ea-84c9-4d89-88a7-90bb15fa5ccb None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:30.039416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:30.142279 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 519/1051] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:43:29 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 23:43:30.147907 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f9a08-0316-49e3-b19c-7fa948f37b52 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:30.219039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd662f-811a-4cd2-87e7-37f8938bf3ef None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:30.219219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd662f-811a-4cd2-87e7-37f8938bf3ef None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:30.219504 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd662f-811a-4cd2-87e7-37f8938bf3ef None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:30.220983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 520/1052] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:43:30.225756 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28e562-f39d-40a6-a1ad-bbf7319623c3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:30.226036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28e562-f39d-40a6-a1ad-bbf7319623c3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:30.226304 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28e562-f39d-40a6-a1ad-bbf7319623c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:30.243547 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:30.245942 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0a28e562-f39d-40a6-a1ad-bbf7319623c3 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:30.247223 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0a28e562-f39d-40a6-a1ad-bbf7319623c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:30.250065 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f9a08-0316-49e3-b19c-7fa948f37b52 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:30.285130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01102815-c5ba-4f8c-801a-859d3582fce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:30.285766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01102815-c5ba-4f8c-801a-859d3582fce7 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:30.286044 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01102815-c5ba-4f8c-801a-859d3582fce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:30.286308 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01102815-c5ba-4f8c-801a-859d3582fce7 None admin] PATH_INFO: `/v3/projects/501bb76cd60a4b6694c18df2bc9311e5` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:30.287468 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01102815-c5ba-4f8c-801a-859d3582fce7 None admin] RBAC: Authorizing `identity:get_project(project_id=501bb76cd60a4b6694c18df2bc9311e5)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:30.288886 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01102815-c5ba-4f8c-801a-859d3582fce7 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:30.294690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 533/1053] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:43:30 2022] GET /identity/v3/projects/501bb76cd60a4b6694c18df2bc9311e5 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:30.298002 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01102815-c5ba-4f8c-801a-859d3582fce7 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:30.328164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 521/1054] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:30 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 23:43:30.405689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01102815-c5ba-4f8c-801a-859d3582fce7 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:30.441000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de4967-dc2c-48d4-9122-6fd4d3181b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:30.441658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de4967-dc2c-48d4-9122-6fd4d3181b02 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:30.441941 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de4967-dc2c-48d4-9122-6fd4d3181b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:30.442213 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de4967-dc2c-48d4-9122-6fd4d3181b02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:30.442961 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de4967-dc2c-48d4-9122-6fd4d3181b02 None admin] RBAC: Adding query filter params (role.id=9dce48a2b53c482d931dcd064b07744e, scope.project.id=501bb76cd60a4b6694c18df2bc9311e5, user.id=3491509777d44f7eb4e99fe21bf50133) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:30.443269 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de4967-dc2c-48d4-9122-6fd4d3181b02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:30.444693 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de4967-dc2c-48d4-9122-6fd4d3181b02 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:30.458937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 534/1055] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:43:30 2022] GET /identity/v3/role_assignments?user.id=3491509777d44f7eb4e99fe21bf50133&scope.project.id=501bb76cd60a4b6694c18df2bc9311e5&role.id=9dce48a2b53c482d931dcd064b07744e => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:30.604358 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28e562-f39d-40a6-a1ad-bbf7319623c3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:30.640293 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:30.710291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28e562-f39d-40a6-a1ad-bbf7319623c3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:30.725662 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2afbc-448a-4319-9d56-0e3e419826fa None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:30.725938 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2afbc-448a-4319-9d56-0e3e419826fa None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:30.726210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2afbc-448a-4319-9d56-0e3e419826fa None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:30.727768 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 535/1056] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:43:30.732652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62356f46-6385-4580-a709-45ef4adbf5ce None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:30.732928 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62356f46-6385-4580-a709-45ef4adbf5ce None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:30.733199 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62356f46-6385-4580-a709-45ef4adbf5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:30.746805 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dbb7fe-3ea1-4962-afbd-58996e3a5506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:30.747436 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dbb7fe-3ea1-4962-afbd-58996e3a5506 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:30.747716 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dbb7fe-3ea1-4962-afbd-58996e3a5506 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:30.747985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dbb7fe-3ea1-4962-afbd-58996e3a5506 None admin] PATH_INFO: `/v3/services/image` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:30.748567 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dbb7fe-3ea1-4962-afbd-58996e3a5506 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:30.749943 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dbb7fe-3ea1-4962-afbd-58996e3a5506 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:30.753131 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-62356f46-6385-4580-a709-45ef4adbf5ce None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:30.754360 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-62356f46-6385-4580-a709-45ef4adbf5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:30.759052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-79dbb7fe-3ea1-4962-afbd-58996e3a5506 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 23:43:30.759052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:30.759052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 23:43:30.759052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 23:43:30.759052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 23:43:30.759052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 23:43:30.759052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 23:43:30.759052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:30.759052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 23:43:30.759052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:30.759052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:43:30.759052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:30.759052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:30.759052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:30.759052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:30.759052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:30.759052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:30.760602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:30.760602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:30.760602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:30.760602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:30.760602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:30.760602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:30.760602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 23:43:30.760602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 23:43:30.760602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 23:43:30.760602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 23:43:30.760602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:30.760602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:30.760602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:30.760602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:30.760602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:30.760602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:30.761452 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:30.761452 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:30.761452 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:30.761452 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:30.761452 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:30.761452 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:30.761452 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:30.761452 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:30.761452 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:30.761452 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:30.761452 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 23:43:30.761452 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:43:30.761452 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 23:43:30.761452 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:30.763233 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 522/1057] 10.222.0.17 () {56 vars in 1105 bytes} [Sat Sep 10 23:43:30 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:30.766981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dbb7fe-3ea1-4962-afbd-58996e3a5506 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:30.779701 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:30.823994 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 536/1058] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:30 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 23:43:30.865688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:30.872250 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dbb7fe-3ea1-4962-afbd-58996e3a5506 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:30.906572 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3ab00-8695-4c45-9683-ea1f4be57d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:30.907193 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3ab00-8695-4c45-9683-ea1f4be57d51 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:30.907468 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3ab00-8695-4c45-9683-ea1f4be57d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:30.907733 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3ab00-8695-4c45-9683-ea1f4be57d51 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:30.908348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c3ab00-8695-4c45-9683-ea1f4be57d51 None admin] RBAC: Adding query filter params (name=image) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:30.908639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c3ab00-8695-4c45-9683-ea1f4be57d51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:30.910048 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c3ab00-8695-4c45-9683-ea1f4be57d51 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:30.915635 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 523/1059] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:43:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:30.921939 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62356f46-6385-4580-a709-45ef4adbf5ce None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:31.023139 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62356f46-6385-4580-a709-45ef4adbf5ce None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:31.056247 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0c107-8f75-4002-92fc-42c20caae9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:31.057010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0c107-8f75-4002-92fc-42c20caae9a5 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:31.057539 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0c107-8f75-4002-92fc-42c20caae9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:31.057802 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0c107-8f75-4002-92fc-42c20caae9a5 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:31.058417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0c107-8f75-4002-92fc-42c20caae9a5 None admin] RBAC: Adding query filter params (type=image) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:31.058735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0c107-8f75-4002-92fc-42c20caae9a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:31.060077 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0c107-8f75-4002-92fc-42c20caae9a5 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:31.070077 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 537/1060] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:43:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:31.071057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3ab00-8695-4c45-9683-ea1f4be57d51 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:31.099292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0c107-8f75-4002-92fc-42c20caae9a5 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:31.124825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:31.179626 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3ab00-8695-4c45-9683-ea1f4be57d51 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:31.204005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0c107-8f75-4002-92fc-42c20caae9a5 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:31.215469 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5e107-502d-4347-8a25-4992dc912cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:31.215821 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5e107-502d-4347-8a25-4992dc912cef None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:31.216099 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5e107-502d-4347-8a25-4992dc912cef None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:31.216369 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5e107-502d-4347-8a25-4992dc912cef None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:31.217036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f5e107-502d-4347-8a25-4992dc912cef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:31.218399 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f5e107-502d-4347-8a25-4992dc912cef None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:31.240034 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c867b-2bd7-414c-934d-40b204ea40e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:31.240660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c867b-2bd7-414c-934d-40b204ea40e0 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:31.240936 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c867b-2bd7-414c-934d-40b204ea40e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:31.241202 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c867b-2bd7-414c-934d-40b204ea40e0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:31.251949 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c867b-2bd7-414c-934d-40b204ea40e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:43:31.256689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-707c867b-2bd7-414c-934d-40b204ea40e0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 23:43:31.256689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:31.256689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:31.256689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:31.256689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:31.256689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:31.256689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:31.256689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:31.256689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:31.256689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:31.256689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:31.256689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:31.256689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:43:31.256689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:43:31.256689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:43:31.256689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:43:31.257783 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:43:31.257783 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:43:31.257783 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:43:31.257783 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:43:31.257783 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:31.257783 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:31.257783 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:31.257783 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:31.257783 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:31.257783 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:31.257783 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:31.257783 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:31.257783 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:31.257783 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:31.257783 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:31.257783 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:31.258818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:31.258818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:31.258818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:31.258818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:31.258818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:43:31.258818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:43:31.258818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:43:31.258818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:43:31.258818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:43:31.258818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:43:31.258818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 23:43:31.258818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:31.262822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 538/1061] 10.222.0.17 () {56 vars in 1120 bytes} [Sat Sep 10 23:43:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:43:31.263638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c867b-2bd7-414c-934d-40b204ea40e0 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:31.298165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 524/1062] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Sep 10 23:43:31 2022] POST /identity/v3/endpoints => generated 327 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:43:31.369162 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:31.389265 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c867b-2bd7-414c-934d-40b204ea40e0 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:31.434433 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f098403-6734-40b3-8f51-a0a3ad5c1a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:31.435140 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f098403-6734-40b3-8f51-a0a3ad5c1a93 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:31.435409 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f098403-6734-40b3-8f51-a0a3ad5c1a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:31.435592 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f098403-6734-40b3-8f51-a0a3ad5c1a93 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:31.436319 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f098403-6734-40b3-8f51-a0a3ad5c1a93 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:31.436619 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f098403-6734-40b3-8f51-a0a3ad5c1a93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:31.438005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f098403-6734-40b3-8f51-a0a3ad5c1a93 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:31.452000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 539/1063] 10.222.0.17 () {56 vars in 1115 bytes} [Sat Sep 10 23:43:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:31.453042 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5e107-502d-4347-8a25-4992dc912cef None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:31.473373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:31.559528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5e107-502d-4347-8a25-4992dc912cef None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:31.595005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c91dd-61e8-4293-965c-b5e3a4305566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:31.595866 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c91dd-61e8-4293-965c-b5e3a4305566 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:31.596302 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c91dd-61e8-4293-965c-b5e3a4305566 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:31.596714 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c91dd-61e8-4293-965c-b5e3a4305566 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:31.607434 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c91dd-61e8-4293-965c-b5e3a4305566 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:31.609062 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c91dd-61e8-4293-965c-b5e3a4305566 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:31.623883 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-b64c91dd-61e8-4293-965c-b5e3a4305566 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:31.623883 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:31.623883 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:31.623883 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:31.623883 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:31.623883 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:31.623883 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:31.623883 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:31.623883 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:31.623883 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:31.623883 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:31.623883 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:31.623883 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:43:31.623883 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:43:31.623883 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:43:31.623883 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:43:31.625025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:31.625025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:31.625025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:31.625025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:31.625025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:31.625025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:31.625025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:31.625025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:31.625025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:31.625025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:31.625025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:31.625025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:31.625025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:31.625025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:31.625025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:31.625025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:31.625025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:43:31.626122 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:43:31.626122 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:31.626122 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:31.627789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 525/1064] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:43:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:31.633603 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f098403-6734-40b3-8f51-a0a3ad5c1a93 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:31.736691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f098403-6734-40b3-8f51-a0a3ad5c1a93 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:31.770456 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673b40f-b8ec-4a2a-b430-a5f2fad9e5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:31.771083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673b40f-b8ec-4a2a-b430-a5f2fad9e5b5 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:31.771358 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673b40f-b8ec-4a2a-b430-a5f2fad9e5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:31.771626 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673b40f-b8ec-4a2a-b430-a5f2fad9e5b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:31.772263 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2673b40f-b8ec-4a2a-b430-a5f2fad9e5b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:31.772563 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2673b40f-b8ec-4a2a-b430-a5f2fad9e5b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:31.773909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2673b40f-b8ec-4a2a-b430-a5f2fad9e5b5 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:31.782988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:31.786956 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 540/1065] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:31.793092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c91dd-61e8-4293-965c-b5e3a4305566 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:31.862179 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:31.896447 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c91dd-61e8-4293-965c-b5e3a4305566 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:31.929601 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a73b64f-7318-40eb-8653-264c3a817516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:31.930217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73b64f-7318-40eb-8653-264c3a817516 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:31.930493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73b64f-7318-40eb-8653-264c3a817516 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:31.930819 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73b64f-7318-40eb-8653-264c3a817516 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:31.939068 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a73b64f-7318-40eb-8653-264c3a817516 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:31.940507 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a73b64f-7318-40eb-8653-264c3a817516 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:31.951457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-4a73b64f-7318-40eb-8653-264c3a817516 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 23:43:31.951457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:31.951457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:31.951457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:31.951457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:31.951457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:31.951457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:31.951457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:31.951457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:31.951457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:31.951457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:31.951457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:31.951457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:43:31.951457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:43:31.951457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:43:31.951457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:43:31.952376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:31.952376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:31.952376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:43:31.952376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:43:31.952376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:43:31.952376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:43:31.952376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:31.952376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:31.952376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:31.952376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:31.952376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:31.952376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:31.952376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:31.952376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:31.952376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:31.952376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:31.952376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:31.952376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:31.953308 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:31.953308 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:31.953308 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:43:31.953308 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:43:31.953308 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:43:31.953308 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:43:31.953308 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:43:31.953308 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:43:31.953308 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 23:43:31.953308 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:31.954822 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 526/1066] 10.222.0.17 () {56 vars in 1093 bytes} [Sat Sep 10 23:43:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:31.958254 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673b40f-b8ec-4a2a-b430-a5f2fad9e5b5 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:32.065968 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673b40f-b8ec-4a2a-b430-a5f2fad9e5b5 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:32.100282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5fc1a7-62a2-450f-a173-71be71c42b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:32.101181 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fc1a7-62a2-450f-a173-71be71c42b59 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:32.101633 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fc1a7-62a2-450f-a173-71be71c42b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:32.102104 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fc1a7-62a2-450f-a173-71be71c42b59 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:32.103026 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5fc1a7-62a2-450f-a173-71be71c42b59 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:32.103537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5fc1a7-62a2-450f-a173-71be71c42b59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:32.105847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5fc1a7-62a2-450f-a173-71be71c42b59 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:32.127998 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 541/1067] 10.222.0.17 () {56 vars in 1133 bytes} [Sat Sep 10 23:43:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:32.129262 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a73b64f-7318-40eb-8653-264c3a817516 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:32.233272 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a73b64f-7318-40eb-8653-264c3a817516 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:32.260552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:32.268924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58537d5a-5dbe-4348-aaef-094719f30e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:32.269612 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58537d5a-5dbe-4348-aaef-094719f30e35 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:32.269887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58537d5a-5dbe-4348-aaef-094719f30e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:32.270160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58537d5a-5dbe-4348-aaef-094719f30e35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:32.281858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58537d5a-5dbe-4348-aaef-094719f30e35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:32.283342 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58537d5a-5dbe-4348-aaef-094719f30e35 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:32.297986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-58537d5a-5dbe-4348-aaef-094719f30e35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:32.297986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:32.297986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:32.297986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:32.297986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:32.297986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:32.297986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:32.297986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:32.297986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:32.297986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:32.297986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:32.297986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:32.297986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:43:32.297986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:43:32.297986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:43:32.297986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:43:32.299226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:32.299226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:32.299226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:32.299226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:32.299226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:32.299226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:32.299226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:32.299226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:32.299226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:32.299226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:32.299226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:32.299226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:32.299226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:32.299226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:32.299226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:32.299226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:32.299226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:43:32.300421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:43:32.300421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:32.300421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:32.304359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 527/1068] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:43:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:43:32.306580 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5fc1a7-62a2-450f-a173-71be71c42b59 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:32.366858 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:32.408076 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5fc1a7-62a2-450f-a173-71be71c42b59 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:32.445457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7c9ec-1457-493d-90b9-c88957b18488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:32.446222 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7c9ec-1457-493d-90b9-c88957b18488 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:32.446587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7c9ec-1457-493d-90b9-c88957b18488 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:32.446974 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7c9ec-1457-493d-90b9-c88957b18488 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:32.447765 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b7c9ec-1457-493d-90b9-c88957b18488 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:32.448148 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b7c9ec-1457-493d-90b9-c88957b18488 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:32.449640 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b7c9ec-1457-493d-90b9-c88957b18488 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:32.465768 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 542/1069] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:32.467158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58537d5a-5dbe-4348-aaef-094719f30e35 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:32.575461 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58537d5a-5dbe-4348-aaef-094719f30e35 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:32.609822 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778ccc3-32d7-4292-b470-b9fc93718f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:32.610450 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778ccc3-32d7-4292-b470-b9fc93718f92 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:32.610750 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778ccc3-32d7-4292-b470-b9fc93718f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:32.611019 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778ccc3-32d7-4292-b470-b9fc93718f92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:32.617929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778ccc3-32d7-4292-b470-b9fc93718f92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:32.619784 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778ccc3-32d7-4292-b470-b9fc93718f92 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:32.631064 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-b778ccc3-32d7-4292-b470-b9fc93718f92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:43:32.631064 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:32.631064 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:32.631064 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:32.631064 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:32.631064 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:32.631064 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:32.631064 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:32.631064 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:32.631064 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:32.631064 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:32.631064 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:32.631064 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:43:32.631064 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:43:32.631064 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:43:32.632000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:43:32.632000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:32.632000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:32.632000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:32.632000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:32.632000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:32.632000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:32.632000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:32.632000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:32.632000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:32.632000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:32.632000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:32.632000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:32.632000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:32.632000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:32.632000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:32.632000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:32.632895 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:43:32.632895 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:43:32.632895 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:43:32.632895 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:43:32.632895 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:43:32.632895 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:43:32.632895 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:43:32.632895 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:32.635540 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 528/1070] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:43:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:32.638461 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7c9ec-1457-493d-90b9-c88957b18488 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:32.649394 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce885a0-fd71-437c-8d58-5de442604e19 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:32.649679 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce885a0-fd71-437c-8d58-5de442604e19 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:32.649960 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce885a0-fd71-437c-8d58-5de442604e19 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:32.652668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 529/1071] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:43:32.657723 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b502932-f065-4376-bfc2-f7022cf47a69 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:32.658002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b502932-f065-4376-bfc2-f7022cf47a69 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:32.658273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b502932-f065-4376-bfc2-f7022cf47a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:32.678234 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-4b502932-f065-4376-bfc2-f7022cf47a69 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:32.679454 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4b502932-f065-4376-bfc2-f7022cf47a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:32.720469 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:32.747659 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7c9ec-1457-493d-90b9-c88957b18488 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:32.757357 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 530/1072] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 23:43:32.766742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:32.782942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7474c9-f229-4d09-bed7-8a24e8d4d46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:32.783686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7474c9-f229-4d09-bed7-8a24e8d4d46a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:32.784060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7474c9-f229-4d09-bed7-8a24e8d4d46a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:32.784420 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7474c9-f229-4d09-bed7-8a24e8d4d46a None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:32.785198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7474c9-f229-4d09-bed7-8a24e8d4d46a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:32.785600 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7474c9-f229-4d09-bed7-8a24e8d4d46a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:32.787136 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7474c9-f229-4d09-bed7-8a24e8d4d46a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:32.804617 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 543/1073] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:43:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:32.805866 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b502932-f065-4376-bfc2-f7022cf47a69 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:32.907519 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b502932-f065-4376-bfc2-f7022cf47a69 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:32.942725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf409b2-f57b-43aa-8c68-1a9a3621e66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:32.943513 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf409b2-f57b-43aa-8c68-1a9a3621e66e None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:32.943890 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf409b2-f57b-43aa-8c68-1a9a3621e66e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:32.944261 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf409b2-f57b-43aa-8c68-1a9a3621e66e None admin] PATH_INFO: `/v3/projects/947277c4458645899287e6818947c076/users/636bfafb693044d283530063047fc0c2/roles/24493ac356254954854aeec4ad5717d5` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:32.946515 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf409b2-f57b-43aa-8c68-1a9a3621e66e None admin] RBAC: Authorizing `identity:create_grant(project_id=947277c4458645899287e6818947c076, user_id=636bfafb693044d283530063047fc0c2, role_id=24493ac356254954854aeec4ad5717d5)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:32.948477 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf409b2-f57b-43aa-8c68-1a9a3621e66e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:33.014691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 531/1074] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Sep 10 23:43:32 2022] PUT /identity/v3/projects/947277c4458645899287e6818947c076/users/636bfafb693044d283530063047fc0c2/roles/24493ac356254954854aeec4ad5717d5 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:43:33.026277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7474c9-f229-4d09-bed7-8a24e8d4d46a None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:33.140674 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7474c9-f229-4d09-bed7-8a24e8d4d46a None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:33.181444 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8b156-5d9e-4742-a992-d6718936f3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:33.183179 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8b156-5d9e-4742-a992-d6718936f3fa None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:33.183448 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8b156-5d9e-4742-a992-d6718936f3fa None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:33.183706 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8b156-5d9e-4742-a992-d6718936f3fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:33.197459 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8b156-5d9e-4742-a992-d6718936f3fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:43:33.202344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-ecd8b156-5d9e-4742-a992-d6718936f3fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:43:33.202344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:33.202344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:33.202344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:33.202344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:33.202344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:33.202344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:33.202344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:33.202344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:33.202344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:33.202344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:33.202344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:33.202344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:43:33.202344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:43:33.202344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:43:33.202344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:43:33.203275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:43:33.203275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:43:33.203275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:43:33.203275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:43:33.203275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:33.203275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:33.203275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:33.203275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:33.203275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:33.203275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:33.203275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:33.203275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:33.203275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:33.203275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:33.203275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:33.203275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:33.204117 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:33.204117 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:33.204117 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:33.204117 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:33.204117 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:43:33.204117 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:43:33.204117 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:43:33.204117 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:43:33.204117 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:43:33.204117 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:43:33.204117 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:43:33.204117 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:33.204117 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 544/1075] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:43:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:33.209597 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf409b2-f57b-43aa-8c68-1a9a3621e66e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:33.241791 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:33.315946 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf409b2-f57b-43aa-8c68-1a9a3621e66e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:33.351423 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf287df8-afa7-4084-b065-ae70d4e85c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:33.352025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf287df8-afa7-4084-b065-ae70d4e85c9f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:33.352302 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf287df8-afa7-4084-b065-ae70d4e85c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:33.352570 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf287df8-afa7-4084-b065-ae70d4e85c9f None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:33.353227 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf287df8-afa7-4084-b065-ae70d4e85c9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:33.353561 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf287df8-afa7-4084-b065-ae70d4e85c9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:33.354977 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf287df8-afa7-4084-b065-ae70d4e85c9f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:33.363695 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 532/1076] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:43:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:33.369409 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8b156-5d9e-4742-a992-d6718936f3fa None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:33.412244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:33.474432 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8b156-5d9e-4742-a992-d6718936f3fa None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:33.510844 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0270c36-107a-48e0-898a-3bafe3ebdc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:33.510844 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0270c36-107a-48e0-898a-3bafe3ebdc42 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:33.510844 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0270c36-107a-48e0-898a-3bafe3ebdc42 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:33.510844 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0270c36-107a-48e0-898a-3bafe3ebdc42 None admin] PATH_INFO: `/v3/users/3491509777d44f7eb4e99fe21bf50133` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:33.515384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0270c36-107a-48e0-898a-3bafe3ebdc42 None admin] RBAC: Authorizing `identity:get_user(user_id=3491509777d44f7eb4e99fe21bf50133)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:33.516837 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0270c36-107a-48e0-898a-3bafe3ebdc42 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:33.518622 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 545/1077] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:43:33 2022] GET /identity/v3/users/3491509777d44f7eb4e99fe21bf50133 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:33.524270 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf287df8-afa7-4084-b065-ae70d4e85c9f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:33.533821 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada140c2-1a6a-487f-83ea-31de7701dcfe None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:33.534385 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada140c2-1a6a-487f-83ea-31de7701dcfe None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:33.534942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada140c2-1a6a-487f-83ea-31de7701dcfe None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:33.536710 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 546/1078] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:43:33.541939 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3073a16-b4c6-47eb-8a67-8f7790eb6f55 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:33.542561 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3073a16-b4c6-47eb-8a67-8f7790eb6f55 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:33.542860 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3073a16-b4c6-47eb-8a67-8f7790eb6f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:33.565063 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b3073a16-b4c6-47eb-8a67-8f7790eb6f55 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:33.566236 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b3073a16-b4c6-47eb-8a67-8f7790eb6f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:33.625078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:33.633900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf287df8-afa7-4084-b065-ae70d4e85c9f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:33.640859 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 547/1079] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 23:43:33.670720 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2212bf-a81d-4fdf-b15e-6ae2d1aa1cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:33.671741 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2212bf-a81d-4fdf-b15e-6ae2d1aa1cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:33.672271 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2212bf-a81d-4fdf-b15e-6ae2d1aa1cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:33.672859 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2212bf-a81d-4fdf-b15e-6ae2d1aa1cc3 None admin] PATH_INFO: `/v3/projects/501bb76cd60a4b6694c18df2bc9311e5` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:33.674226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2212bf-a81d-4fdf-b15e-6ae2d1aa1cc3 None admin] RBAC: Authorizing `identity:get_project(project_id=501bb76cd60a4b6694c18df2bc9311e5)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:33.675948 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2212bf-a81d-4fdf-b15e-6ae2d1aa1cc3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:33.678123 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 533/1080] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:43:33 2022] GET /identity/v3/projects/501bb76cd60a4b6694c18df2bc9311e5 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:33.684842 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3073a16-b4c6-47eb-8a67-8f7790eb6f55 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:33.721696 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:33.787043 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3073a16-b4c6-47eb-8a67-8f7790eb6f55 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:33.820896 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672538a-328c-4923-81ec-be2559a2257b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:33.821552 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672538a-328c-4923-81ec-be2559a2257b None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:33.821828 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672538a-328c-4923-81ec-be2559a2257b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:33.822097 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672538a-328c-4923-81ec-be2559a2257b None admin] PATH_INFO: `/v3/projects/501bb76cd60a4b6694c18df2bc9311e5/users/3491509777d44f7eb4e99fe21bf50133/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:33.824267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4672538a-328c-4923-81ec-be2559a2257b None admin] RBAC: Authorizing `identity:create_grant(project_id=501bb76cd60a4b6694c18df2bc9311e5, user_id=3491509777d44f7eb4e99fe21bf50133, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:33.825719 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4672538a-328c-4923-81ec-be2559a2257b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:33.914815 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2212bf-a81d-4fdf-b15e-6ae2d1aa1cc3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:33.924136 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 548/1081] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Sep 10 23:43:33 2022] PUT /identity/v3/projects/501bb76cd60a4b6694c18df2bc9311e5/users/3491509777d44f7eb4e99fe21bf50133/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 23:43:34.033680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2212bf-a81d-4fdf-b15e-6ae2d1aa1cc3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:34.070731 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefdd61-35b7-436a-abad-93e9e7a438c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:34.071273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefdd61-35b7-436a-abad-93e9e7a438c3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:34.071548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefdd61-35b7-436a-abad-93e9e7a438c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:34.071816 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefdd61-35b7-436a-abad-93e9e7a438c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:34.102299 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:34.145004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefdd61-35b7-436a-abad-93e9e7a438c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:34.146461 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefdd61-35b7-436a-abad-93e9e7a438c3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:34.249677 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 534/1082] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:43:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 10 23:43:34.255248 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672538a-328c-4923-81ec-be2559a2257b None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:34.356478 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:34.362708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672538a-328c-4923-81ec-be2559a2257b None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:34.397558 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb21f26-fe65-46fe-ad5f-371e6236104c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:34.398176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb21f26-fe65-46fe-ad5f-371e6236104c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:34.398453 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb21f26-fe65-46fe-ad5f-371e6236104c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:34.398759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb21f26-fe65-46fe-ad5f-371e6236104c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:34.408776 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb21f26-fe65-46fe-ad5f-371e6236104c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:34.410206 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb21f26-fe65-46fe-ad5f-371e6236104c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:34.425233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-4bb21f26-fe65-46fe-ad5f-371e6236104c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:34.425233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:34.425233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:34.425233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:34.425233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:34.425233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:34.425233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:34.425233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:34.425233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:34.425233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:34.425233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:34.425233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:34.425233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:43:34.425233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:43:34.425233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:43:34.425233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:43:34.426215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:34.426215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:34.426215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:34.426215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:34.426215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:34.426215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:34.426215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:34.426215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:34.426215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:34.426215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:34.426215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:34.426215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:34.426215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:34.426215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:34.426215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:34.426215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:34.426215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:43:34.427142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:43:34.427142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:34.427142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:34.431024 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 549/1083] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:43:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:34.433076 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefdd61-35b7-436a-abad-93e9e7a438c3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:34.493488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:34.535700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefdd61-35b7-436a-abad-93e9e7a438c3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:34.569611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47003b04-3c75-48e6-98f9-cff3439eb0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:34.570206 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47003b04-3c75-48e6-98f9-cff3439eb0e3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:34.570484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47003b04-3c75-48e6-98f9-cff3439eb0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:34.570789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47003b04-3c75-48e6-98f9-cff3439eb0e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:34.571422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47003b04-3c75-48e6-98f9-cff3439eb0e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:34.571720 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47003b04-3c75-48e6-98f9-cff3439eb0e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:34.573655 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47003b04-3c75-48e6-98f9-cff3439eb0e3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:34.585711 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 535/1084] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:43:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:35.229955 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d930a-558c-4642-a311-9dbdbefd2efe None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:35.230593 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d930a-558c-4642-a311-9dbdbefd2efe None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:35.230593 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d930a-558c-4642-a311-9dbdbefd2efe None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:35.231948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 550/1085] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:35.237335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea26fc1-f3b1-4552-a4e6-a5354d2dca32 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:35.237619 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea26fc1-f3b1-4552-a4e6-a5354d2dca32 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:35.237895 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea26fc1-f3b1-4552-a4e6-a5354d2dca32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:35.256343 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-aea26fc1-f3b1-4552-a4e6-a5354d2dca32 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:35.257529 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-aea26fc1-f3b1-4552-a4e6-a5354d2dca32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:35.324768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 536/1086] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:43:35.599426 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d930a-558c-4642-a311-9dbdbefd2efe None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:35.707546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d930a-558c-4642-a311-9dbdbefd2efe None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:35.740379 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3890055c-f695-4570-b9f2-7108d15e7f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:35.741002 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3890055c-f695-4570-b9f2-7108d15e7f9d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:35.741280 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3890055c-f695-4570-b9f2-7108d15e7f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:35.741549 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3890055c-f695-4570-b9f2-7108d15e7f9d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:35.752544 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3890055c-f695-4570-b9f2-7108d15e7f9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:43:35.757290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-3890055c-f695-4570-b9f2-7108d15e7f9d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 23:43:35.757290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:35.757290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:35.757290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:35.757290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:35.757290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:35.757290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:35.757290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:35.757290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:35.757290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:35.757290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:35.757290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:35.757290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:43:35.757290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:43:35.757290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:43:35.757290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:43:35.758427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:43:35.758427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:43:35.758427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:43:35.758427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:43:35.758427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:35.758427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:35.758427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:35.758427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:35.758427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:35.758427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:35.758427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:35.758427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:35.758427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:35.758427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:35.758427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:35.758427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:35.759372 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:35.759372 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:35.759372 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:35.759372 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:35.759372 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:43:35.759372 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:43:35.759372 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:43:35.759372 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:43:35.759372 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:43:35.759372 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:43:35.759372 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 23:43:35.759372 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:35.759372 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 551/1087] 10.222.0.17 () {56 vars in 1120 bytes} [Sat Sep 10 23:43:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:35.764983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea26fc1-f3b1-4552-a4e6-a5354d2dca32 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:35.813843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:35.861692 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea26fc1-f3b1-4552-a4e6-a5354d2dca32 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:35.895049 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a220d1d-4122-4b38-b5f6-830ee0125d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:35.895643 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a220d1d-4122-4b38-b5f6-830ee0125d90 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:35.895919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a220d1d-4122-4b38-b5f6-830ee0125d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:35.896187 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a220d1d-4122-4b38-b5f6-830ee0125d90 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:35.896836 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a220d1d-4122-4b38-b5f6-830ee0125d90 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:35.897134 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a220d1d-4122-4b38-b5f6-830ee0125d90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:35.898496 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a220d1d-4122-4b38-b5f6-830ee0125d90 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:35.906171 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 537/1088] 10.222.0.17 () {56 vars in 1115 bytes} [Sat Sep 10 23:43:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:35.911959 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3890055c-f695-4570-b9f2-7108d15e7f9d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:35.922957 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:36.019226 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3890055c-f695-4570-b9f2-7108d15e7f9d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:36.053818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ddd198-4c14-4e45-8850-8c6244efa63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:36.054438 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddd198-4c14-4e45-8850-8c6244efa63a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:36.054738 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddd198-4c14-4e45-8850-8c6244efa63a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:36.055009 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddd198-4c14-4e45-8850-8c6244efa63a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:36.066111 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ddd198-4c14-4e45-8850-8c6244efa63a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:36.067571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ddd198-4c14-4e45-8850-8c6244efa63a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:36.081831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-33ddd198-4c14-4e45-8850-8c6244efa63a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:36.081831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:36.081831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:36.081831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:36.081831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:36.081831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:36.081831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:36.081831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:36.081831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:36.081831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:36.081831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:36.081831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:36.081831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:43:36.081831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:43:36.081831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:43:36.081831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:43:36.083005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:36.083005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:36.083005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:36.083005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:36.083005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:36.083005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:36.083005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:36.083005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:36.083005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:36.083005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:36.083005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:36.083005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:36.083005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:36.083005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:36.083005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:36.083005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:36.083005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:43:36.084118 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:43:36.084118 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:36.084118 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:36.086810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 552/1089] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:43:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:36.088642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a220d1d-4122-4b38-b5f6-830ee0125d90 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:36.117111 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f5dfd-b2ff-4ac4-9092-ce80f4760580 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:36.117314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f5dfd-b2ff-4ac4-9092-ce80f4760580 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:36.117610 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f5dfd-b2ff-4ac4-9092-ce80f4760580 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:36.119040 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 553/1090] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:43:36.124624 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae52e90-8380-4f7f-80d1-d977126e8802 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:36.124903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae52e90-8380-4f7f-80d1-d977126e8802 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:36.125182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae52e90-8380-4f7f-80d1-d977126e8802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:36.142914 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-eae52e90-8380-4f7f-80d1-d977126e8802 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:36.144098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-eae52e90-8380-4f7f-80d1-d977126e8802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:36.189084 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a220d1d-4122-4b38-b5f6-830ee0125d90 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:36.199700 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:36.201937 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:36.210889 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 554/1091] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 23:43:36.223172 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79f08e-f2ac-4a7f-adad-aa814fa912e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:36.225375 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79f08e-f2ac-4a7f-adad-aa814fa912e6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:36.225375 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79f08e-f2ac-4a7f-adad-aa814fa912e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:36.225375 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79f08e-f2ac-4a7f-adad-aa814fa912e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:36.225375 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a79f08e-f2ac-4a7f-adad-aa814fa912e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:36.225375 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a79f08e-f2ac-4a7f-adad-aa814fa912e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:36.226565 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a79f08e-f2ac-4a7f-adad-aa814fa912e6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:36.238799 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 538/1092] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:36.244944 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae52e90-8380-4f7f-80d1-d977126e8802 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:36.343503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae52e90-8380-4f7f-80d1-d977126e8802 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:36.376782 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07b800-da67-43dd-ac8b-35496f9ba7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:36.377333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07b800-da67-43dd-ac8b-35496f9ba7d1 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:36.377606 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07b800-da67-43dd-ac8b-35496f9ba7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:36.377872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07b800-da67-43dd-ac8b-35496f9ba7d1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:36.384652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07b800-da67-43dd-ac8b-35496f9ba7d1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:36.386507 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07b800-da67-43dd-ac8b-35496f9ba7d1 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:36.397694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-fc07b800-da67-43dd-ac8b-35496f9ba7d1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 23:43:36.397694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:36.397694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:36.397694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:36.397694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:36.397694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:36.397694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:36.397694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:36.397694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:36.397694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:36.397694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:36.397694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:36.397694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:43:36.397694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:43:36.397694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:43:36.397694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:43:36.398739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:36.398739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:36.398739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:43:36.398739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:43:36.398739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:43:36.398739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:43:36.398739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:36.398739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:36.398739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:36.398739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:36.398739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:36.398739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:36.398739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:36.398739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:36.398739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:36.398739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:36.398739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:36.398739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:36.400197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:36.400197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:36.400197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:43:36.400197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:43:36.400197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:43:36.400197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:43:36.400197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:43:36.400197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:43:36.400197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 23:43:36.400197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:36.400197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 555/1093] 10.222.0.17 () {56 vars in 1093 bytes} [Sat Sep 10 23:43:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:36.404604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79f08e-f2ac-4a7f-adad-aa814fa912e6 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:36.482057 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07b800-da67-43dd-ac8b-35496f9ba7d1 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:36.504340 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79f08e-f2ac-4a7f-adad-aa814fa912e6 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:36.531473 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:36.540075 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c223ec60-06fa-49f7-9d46-7d1331a6c32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:36.540667 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223ec60-06fa-49f7-9d46-7d1331a6c32e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:36.541643 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223ec60-06fa-49f7-9d46-7d1331a6c32e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:36.541772 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223ec60-06fa-49f7-9d46-7d1331a6c32e None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:36.542484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c223ec60-06fa-49f7-9d46-7d1331a6c32e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:36.542829 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c223ec60-06fa-49f7-9d46-7d1331a6c32e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:36.545547 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c223ec60-06fa-49f7-9d46-7d1331a6c32e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:36.561302 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 539/1094] 10.222.0.17 () {56 vars in 1133 bytes} [Sat Sep 10 23:43:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:36.566993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c223ec60-06fa-49f7-9d46-7d1331a6c32e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:36.581567 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07b800-da67-43dd-ac8b-35496f9ba7d1 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:36.596674 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:36.614576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64238b0f-fea3-40c5-8e36-90fa0ada0e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:36.615253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64238b0f-fea3-40c5-8e36-90fa0ada0e0f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:36.615527 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64238b0f-fea3-40c5-8e36-90fa0ada0e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:36.615887 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64238b0f-fea3-40c5-8e36-90fa0ada0e0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:36.627144 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64238b0f-fea3-40c5-8e36-90fa0ada0e0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:43:36.631820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-64238b0f-fea3-40c5-8e36-90fa0ada0e0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:43:36.631820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:36.631820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:36.631820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:36.631820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:36.631820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:36.631820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:36.631820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:36.631820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:36.631820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:36.631820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:36.631820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:36.631820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:43:36.631820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:43:36.631820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:43:36.631820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:43:36.632812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:43:36.632812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:43:36.632812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:43:36.632812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:43:36.632812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:36.632812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:36.632812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:36.632812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:36.632812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:36.632812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:36.632812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:36.632812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:36.632812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:36.632812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:36.632812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:36.632812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:36.633642 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:36.633642 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:36.633642 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:36.633642 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:36.633642 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:43:36.633642 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:43:36.633642 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:43:36.633642 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:43:36.633642 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:43:36.633642 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:43:36.633642 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:43:36.633642 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:36.633642 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 556/1095] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:43:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:43:36.639082 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64238b0f-fea3-40c5-8e36-90fa0ada0e0f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:36.668808 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c223ec60-06fa-49f7-9d46-7d1331a6c32e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:36.702206 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d3b35-8465-46be-8ac8-5332e02aa145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:36.702839 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d3b35-8465-46be-8ac8-5332e02aa145 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:36.703189 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d3b35-8465-46be-8ac8-5332e02aa145 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:36.704332 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d3b35-8465-46be-8ac8-5332e02aa145 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:36.738714 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64238b0f-fea3-40c5-8e36-90fa0ada0e0f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:36.748890 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:36.769997 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4d3b35-8465-46be-8ac8-5332e02aa145 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:36.771474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4d3b35-8465-46be-8ac8-5332e02aa145 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:36.772171 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00adc2-bee1-4467-92a9-8513861f1073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:36.772768 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00adc2-bee1-4467-92a9-8513861f1073 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:36.773044 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00adc2-bee1-4467-92a9-8513861f1073 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:36.773334 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00adc2-bee1-4467-92a9-8513861f1073 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:36.773966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00adc2-bee1-4467-92a9-8513861f1073 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:36.774268 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00adc2-bee1-4467-92a9-8513861f1073 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:36.776363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00adc2-bee1-4467-92a9-8513861f1073 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:36.785863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 557/1096] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:43:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:36.792337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00adc2-bee1-4467-92a9-8513861f1073 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:36.801192 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:36.872108 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 540/1097] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:43:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 10 23:43:36.877867 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d3b35-8465-46be-8ac8-5332e02aa145 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:36.890539 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00adc2-bee1-4467-92a9-8513861f1073 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:36.923891 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf87a43-6eaf-4f96-8921-e4af72089942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:36.924507 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf87a43-6eaf-4f96-8921-e4af72089942 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:36.924776 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf87a43-6eaf-4f96-8921-e4af72089942 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:36.925039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf87a43-6eaf-4f96-8921-e4af72089942 None admin] PATH_INFO: `/v3/users/3491509777d44f7eb4e99fe21bf50133` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:36.926139 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf87a43-6eaf-4f96-8921-e4af72089942 None admin] RBAC: Authorizing `identity:get_user(user_id=3491509777d44f7eb4e99fe21bf50133)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:36.927588 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf87a43-6eaf-4f96-8921-e4af72089942 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:36.929145 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 558/1098] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:43:36 2022] GET /identity/v3/users/3491509777d44f7eb4e99fe21bf50133 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:36.935989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf87a43-6eaf-4f96-8921-e4af72089942 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:36.974938 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d3b35-8465-46be-8ac8-5332e02aa145 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:36.975728 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:37.003111 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:37.008537 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6099f-baa3-4528-9edc-56b121a3c860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:37.009508 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6099f-baa3-4528-9edc-56b121a3c860 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:37.010469 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6099f-baa3-4528-9edc-56b121a3c860 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:37.010790 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6099f-baa3-4528-9edc-56b121a3c860 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:37.021081 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6099f-baa3-4528-9edc-56b121a3c860 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:37.022511 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6099f-baa3-4528-9edc-56b121a3c860 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:37.033280 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf87a43-6eaf-4f96-8921-e4af72089942 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:37.037078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-7db6099f-baa3-4528-9edc-56b121a3c860 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:37.037078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:37.037078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:37.037078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:37.037078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:37.037078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:37.037078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:37.037078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:37.037078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:37.037078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:37.037078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:37.037078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:37.037078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:43:37.037078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:43:37.037078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:43:37.037078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:43:37.038891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:37.038891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:37.038891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:37.038891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:37.038891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:37.038891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:37.038891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:37.038891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:37.038891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:37.038891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:37.038891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:37.038891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:37.038891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:37.038891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:37.038891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:37.038891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:37.038891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:43:37.040711 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:43:37.040711 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:43:37.040711 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:37.040711 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 541/1099] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:43:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:37.045395 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6099f-baa3-4528-9edc-56b121a3c860 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:37.068431 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbda78-f06c-4317-832a-0baddfc01121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:37.069031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbda78-f06c-4317-832a-0baddfc01121 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:37.069350 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbda78-f06c-4317-832a-0baddfc01121 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:37.069623 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbda78-f06c-4317-832a-0baddfc01121 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:37.143288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bbda78-f06c-4317-832a-0baddfc01121 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:37.143909 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6099f-baa3-4528-9edc-56b121a3c860 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:37.144645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bbda78-f06c-4317-832a-0baddfc01121 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:37.177062 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ae054-36b7-4f78-9038-b88a454ab329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:37.177649 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ae054-36b7-4f78-9038-b88a454ab329 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:37.177926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ae054-36b7-4f78-9038-b88a454ab329 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:37.178194 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ae054-36b7-4f78-9038-b88a454ab329 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:37.178842 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ae054-36b7-4f78-9038-b88a454ab329 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:37.179140 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ae054-36b7-4f78-9038-b88a454ab329 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:37.180500 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ae054-36b7-4f78-9038-b88a454ab329 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:37.184454 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:37.192383 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 542/1100] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:37.198878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ae054-36b7-4f78-9038-b88a454ab329 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:37.242127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:37.248344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 559/1101] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:43:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 10 23:43:37.249379 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbda78-f06c-4317-832a-0baddfc01121 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:37.298518 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ae054-36b7-4f78-9038-b88a454ab329 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:37.334654 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d9f82-77dc-4720-aceb-d0c79489a5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:37.335300 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d9f82-77dc-4720-aceb-d0c79489a5db None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:37.335581 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d9f82-77dc-4720-aceb-d0c79489a5db None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:37.335851 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d9f82-77dc-4720-aceb-d0c79489a5db None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:37.343306 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d9f82-77dc-4720-aceb-d0c79489a5db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:37.344744 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d9f82-77dc-4720-aceb-d0c79489a5db None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:37.347245 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbda78-f06c-4317-832a-0baddfc01121 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:37.356291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-1b5d9f82-77dc-4720-aceb-d0c79489a5db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:43:37.356291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:37.356291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:37.356291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:37.356291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:37.356291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:37.356291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:37.356291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:37.356291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:37.356291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:37.356291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:37.356291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:37.356291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:43:37.356291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:43:37.356291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:43:37.357225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:43:37.357225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:37.357225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:37.357225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:37.357225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:37.357225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:37.357225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:37.357225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:37.357225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:37.357225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:37.357225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:37.357225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:37.357225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:37.357225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:37.357225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:37.357225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:37.357225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:37.358322 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:43:37.358322 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:43:37.358322 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:43:37.358322 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:43:37.358322 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:43:37.358322 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:43:37.358322 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:43:37.358322 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:43:37.358322 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 543/1102] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:43:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:43:37.363141 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d9f82-77dc-4720-aceb-d0c79489a5db None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:37.380457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b8807-5890-4c08-ac94-32d25ad09966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:37.381073 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b8807-5890-4c08-ac94-32d25ad09966 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:37.381347 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b8807-5890-4c08-ac94-32d25ad09966 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:37.381612 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b8807-5890-4c08-ac94-32d25ad09966 None admin] PATH_INFO: `/v3/projects/501bb76cd60a4b6694c18df2bc9311e5` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:37.382842 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b8807-5890-4c08-ac94-32d25ad09966 None admin] RBAC: Authorizing `identity:get_project(project_id=501bb76cd60a4b6694c18df2bc9311e5)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:37.384297 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b8807-5890-4c08-ac94-32d25ad09966 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:37.385966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 560/1103] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:43:37 2022] GET /identity/v3/projects/501bb76cd60a4b6694c18df2bc9311e5 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:37.392029 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b8807-5890-4c08-ac94-32d25ad09966 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:37.402529 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:37.460678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d9f82-77dc-4720-aceb-d0c79489a5db None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:37.462204 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:37.489054 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b8807-5890-4c08-ac94-32d25ad09966 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:37.496867 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38b19a-bcb5-4fbc-9824-0c4298d293d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:37.497462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38b19a-bcb5-4fbc-9824-0c4298d293d2 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:37.497739 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38b19a-bcb5-4fbc-9824-0c4298d293d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:37.498010 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38b19a-bcb5-4fbc-9824-0c4298d293d2 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:37.498723 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa38b19a-bcb5-4fbc-9824-0c4298d293d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:37.499026 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa38b19a-bcb5-4fbc-9824-0c4298d293d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:37.500443 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa38b19a-bcb5-4fbc-9824-0c4298d293d2 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:37.511685 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 544/1104] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:43:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:37.517626 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38b19a-bcb5-4fbc-9824-0c4298d293d2 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:37.528378 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1108c9-e27a-4e8a-ae77-1f3f51b851e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:37.528980 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1108c9-e27a-4e8a-ae77-1f3f51b851e3 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:37.529255 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1108c9-e27a-4e8a-ae77-1f3f51b851e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:37.529523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1108c9-e27a-4e8a-ae77-1f3f51b851e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:37.530240 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1108c9-e27a-4e8a-ae77-1f3f51b851e3 None admin] RBAC: Adding query filter params (role.id=9dce48a2b53c482d931dcd064b07744e, scope.project.id=501bb76cd60a4b6694c18df2bc9311e5, user.id=3491509777d44f7eb4e99fe21bf50133) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:37.530541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1108c9-e27a-4e8a-ae77-1f3f51b851e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:37.532009 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1108c9-e27a-4e8a-ae77-1f3f51b851e3 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:37.537624 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 561/1105] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:43:37 2022] GET /identity/v3/role_assignments?user.id=3491509777d44f7eb4e99fe21bf50133&scope.project.id=501bb76cd60a4b6694c18df2bc9311e5&role.id=9dce48a2b53c482d931dcd064b07744e => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:37.620455 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38b19a-bcb5-4fbc-9824-0c4298d293d2 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:37.655463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042a72a-30bd-4b73-a1d6-99a3fb0462b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:37.655985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042a72a-30bd-4b73-a1d6-99a3fb0462b4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:37.656258 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042a72a-30bd-4b73-a1d6-99a3fb0462b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:37.656522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042a72a-30bd-4b73-a1d6-99a3fb0462b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:37.657250 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3042a72a-30bd-4b73-a1d6-99a3fb0462b4 None admin] RBAC: Adding query filter params (role.id=24493ac356254954854aeec4ad5717d5, scope.project.id=947277c4458645899287e6818947c076, user.id=636bfafb693044d283530063047fc0c2) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:37.657546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3042a72a-30bd-4b73-a1d6-99a3fb0462b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:37.659022 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3042a72a-30bd-4b73-a1d6-99a3fb0462b4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:37.665724 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 545/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:43:37 2022] GET /identity/v3/role_assignments?user.id=636bfafb693044d283530063047fc0c2&scope.project.id=947277c4458645899287e6818947c076&role.id=24493ac356254954854aeec4ad5717d5 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:43:39.699710 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740a608-d7cd-42d6-a06a-33d18f2df454 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:39.700234 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740a608-d7cd-42d6-a06a-33d18f2df454 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:39.700234 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740a608-d7cd-42d6-a06a-33d18f2df454 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:39.701577 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 562/1107] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:39.706917 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cace302-97ff-48cf-a9cb-771a9839cba9 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:39.707316 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cace302-97ff-48cf-a9cb-771a9839cba9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:39.707459 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cace302-97ff-48cf-a9cb-771a9839cba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:39.725845 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7cace302-97ff-48cf-a9cb-771a9839cba9 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:39.727061 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7cace302-97ff-48cf-a9cb-771a9839cba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:39.751827 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:39.798318 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 546/1108] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:43:40.076255 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740a608-d7cd-42d6-a06a-33d18f2df454 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:40.146463 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:40.175760 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740a608-d7cd-42d6-a06a-33d18f2df454 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:40.208769 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb1ce3c-e49d-46fb-8bc4-908cc67634b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:40.209374 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1ce3c-e49d-46fb-8bc4-908cc67634b3 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:40.213944 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1ce3c-e49d-46fb-8bc4-908cc67634b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:40.213944 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1ce3c-e49d-46fb-8bc4-908cc67634b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:40.213944 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb1ce3c-e49d-46fb-8bc4-908cc67634b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:40.213944 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb1ce3c-e49d-46fb-8bc4-908cc67634b3 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:40.214729 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 563/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:43:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:40.220409 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cace302-97ff-48cf-a9cb-771a9839cba9 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:40.314325 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cace302-97ff-48cf-a9cb-771a9839cba9 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:40.347854 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f680bf-8107-4e70-ac54-6b48b2540a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:40.348393 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f680bf-8107-4e70-ac54-6b48b2540a29 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:40.348666 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f680bf-8107-4e70-ac54-6b48b2540a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:40.348928 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f680bf-8107-4e70-ac54-6b48b2540a29 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:40.349581 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f680bf-8107-4e70-ac54-6b48b2540a29 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:40.351020 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f680bf-8107-4e70-ac54-6b48b2540a29 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:40.419730 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 547/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Sep 10 23:43:40 2022] POST /identity/v3/users => generated 286 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:43:42.598452 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708c15e-9e32-4372-9fd8-97896d814045 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:42.598988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708c15e-9e32-4372-9fd8-97896d814045 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:42.598988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708c15e-9e32-4372-9fd8-97896d814045 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:42.600262 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 564/1111] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:42.605613 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b32f81-918e-42aa-b0e2-f6399d891285 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:42.606054 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b32f81-918e-42aa-b0e2-f6399d891285 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:42.606146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b32f81-918e-42aa-b0e2-f6399d891285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:42.623850 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c9b32f81-918e-42aa-b0e2-f6399d891285 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:42.625010 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c9b32f81-918e-42aa-b0e2-f6399d891285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:42.632173 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:42.689746 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 548/1112] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:43:42.955498 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708c15e-9e32-4372-9fd8-97896d814045 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:43.060479 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708c15e-9e32-4372-9fd8-97896d814045 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:43.093927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648fa6f3-1409-4023-8d09-70df06f5d757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:43.094542 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648fa6f3-1409-4023-8d09-70df06f5d757 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:43.094846 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648fa6f3-1409-4023-8d09-70df06f5d757 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:43.095115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648fa6f3-1409-4023-8d09-70df06f5d757 None admin] PATH_INFO: `/v3/roles/ff2ed192587843e5b10b5a35949e4c5f` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:43.096482 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648fa6f3-1409-4023-8d09-70df06f5d757 None admin] RBAC: Authorizing `identity:get_role(role_id=ff2ed192587843e5b10b5a35949e4c5f)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:43.097831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648fa6f3-1409-4023-8d09-70df06f5d757 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:43.099196 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 565/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:43:42 2022] GET /identity/v3/roles/ff2ed192587843e5b10b5a35949e4c5f => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:43.105563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b32f81-918e-42aa-b0e2-f6399d891285 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:43.209638 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b32f81-918e-42aa-b0e2-f6399d891285 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:43.243220 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0140d7-2604-4ea2-882e-020cbe218a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:43.243882 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0140d7-2604-4ea2-882e-020cbe218a0f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:43.244402 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0140d7-2604-4ea2-882e-020cbe218a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:43.244508 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0140d7-2604-4ea2-882e-020cbe218a0f None admin] PATH_INFO: `/v3/users/a6931212521a426eae548b2e7bf11584` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:43.255199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:43.257450 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0140d7-2604-4ea2-882e-020cbe218a0f None admin] RBAC: Authorizing `identity:get_user(user_id=a6931212521a426eae548b2e7bf11584)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:43.258903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0140d7-2604-4ea2-882e-020cbe218a0f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:43.261131 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 549/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:43:43 2022] GET /identity/v3/users/a6931212521a426eae548b2e7bf11584 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:43.266865 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648fa6f3-1409-4023-8d09-70df06f5d757 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:43.278139 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:43.368241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648fa6f3-1409-4023-8d09-70df06f5d757 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:43.401358 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2166d0-03b9-4089-a0fd-2b939e0095ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:43.401939 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2166d0-03b9-4089-a0fd-2b939e0095ee None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:43.402208 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2166d0-03b9-4089-a0fd-2b939e0095ee None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:43.402471 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2166d0-03b9-4089-a0fd-2b939e0095ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:43.466616 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2166d0-03b9-4089-a0fd-2b939e0095ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:43.467997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2166d0-03b9-4089-a0fd-2b939e0095ee None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:43.476981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:43.561636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 566/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:43:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 23:43:43.567701 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0140d7-2604-4ea2-882e-020cbe218a0f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:43.678218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0140d7-2604-4ea2-882e-020cbe218a0f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:43.711420 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e636029f-e61a-4630-9d1e-e475a538b8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:43.712028 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636029f-e61a-4630-9d1e-e475a538b8ec None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:43.712304 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636029f-e61a-4630-9d1e-e475a538b8ec None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:43.713103 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636029f-e61a-4630-9d1e-e475a538b8ec None admin] PATH_INFO: `/v3/projects/501bb76cd60a4b6694c18df2bc9311e5` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:43.714692 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e636029f-e61a-4630-9d1e-e475a538b8ec None admin] RBAC: Authorizing `identity:get_project(project_id=501bb76cd60a4b6694c18df2bc9311e5)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:43.716115 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e636029f-e61a-4630-9d1e-e475a538b8ec None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:43.717773 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 550/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:43:43 2022] GET /identity/v3/projects/501bb76cd60a4b6694c18df2bc9311e5 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:43.723801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2166d0-03b9-4089-a0fd-2b939e0095ee None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:43.818402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2166d0-03b9-4089-a0fd-2b939e0095ee None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:43.827218 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:43.850556 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefe72d-9c59-4db3-9723-d18a34e30d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:43.851174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefe72d-9c59-4db3-9723-d18a34e30d6e None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:43.851446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefe72d-9c59-4db3-9723-d18a34e30d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:43.851711 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefe72d-9c59-4db3-9723-d18a34e30d6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:43.852416 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baefe72d-9c59-4db3-9723-d18a34e30d6e None admin] RBAC: Adding query filter params (role.id=ff2ed192587843e5b10b5a35949e4c5f, scope.project.id=501bb76cd60a4b6694c18df2bc9311e5, user.id=a6931212521a426eae548b2e7bf11584) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:43.852713 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baefe72d-9c59-4db3-9723-d18a34e30d6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:43.854705 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baefe72d-9c59-4db3-9723-d18a34e30d6e None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:43.859838 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 567/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:43:43 2022] GET /identity/v3/role_assignments?user.id=a6931212521a426eae548b2e7bf11584&scope.project.id=501bb76cd60a4b6694c18df2bc9311e5&role.id=ff2ed192587843e5b10b5a35949e4c5f => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:45.943560 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8f3d4-28ef-40bb-86f0-cd6f87bb9c92 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:45.944129 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8f3d4-28ef-40bb-86f0-cd6f87bb9c92 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:45.944129 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8f3d4-28ef-40bb-86f0-cd6f87bb9c92 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:45.945427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 551/1118] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:45.951573 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b523e2-2bcd-49ea-b4a2-c0c1d71760cc None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:45.951970 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b523e2-2bcd-49ea-b4a2-c0c1d71760cc None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:45.952124 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b523e2-2bcd-49ea-b4a2-c0c1d71760cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:45.970344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-14b523e2-2bcd-49ea-b4a2-c0c1d71760cc None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:45.972870 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-14b523e2-2bcd-49ea-b4a2-c0c1d71760cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:46.036777 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 568/1119] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:43:46.308661 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8f3d4-28ef-40bb-86f0-cd6f87bb9c92 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:46.373056 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:46.405603 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8f3d4-28ef-40bb-86f0-cd6f87bb9c92 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:46.439486 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba044465-64ec-4b9a-9c80-4440cebb1244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:46.440092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba044465-64ec-4b9a-9c80-4440cebb1244 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:46.440370 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba044465-64ec-4b9a-9c80-4440cebb1244 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:46.440639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba044465-64ec-4b9a-9c80-4440cebb1244 None admin] PATH_INFO: `/v3/roles/ff2ed192587843e5b10b5a35949e4c5f` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:46.442021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba044465-64ec-4b9a-9c80-4440cebb1244 None admin] RBAC: Authorizing `identity:get_role(role_id=ff2ed192587843e5b10b5a35949e4c5f)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:46.443420 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba044465-64ec-4b9a-9c80-4440cebb1244 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:46.444717 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 552/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:43:46 2022] GET /identity/v3/roles/ff2ed192587843e5b10b5a35949e4c5f => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:46.450487 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b523e2-2bcd-49ea-b4a2-c0c1d71760cc None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:46.545391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b523e2-2bcd-49ea-b4a2-c0c1d71760cc None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:46.546903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:46.578558 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59850734-8d30-4fac-ad9f-a010ac0b3105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:46.579145 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59850734-8d30-4fac-ad9f-a010ac0b3105 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:46.579422 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59850734-8d30-4fac-ad9f-a010ac0b3105 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:46.579693 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59850734-8d30-4fac-ad9f-a010ac0b3105 None admin] PATH_INFO: `/v3/users/a6931212521a426eae548b2e7bf11584` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:46.580806 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59850734-8d30-4fac-ad9f-a010ac0b3105 None admin] RBAC: Authorizing `identity:get_user(user_id=a6931212521a426eae548b2e7bf11584)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:46.582227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59850734-8d30-4fac-ad9f-a010ac0b3105 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:46.584821 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 569/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:43:46 2022] GET /identity/v3/users/a6931212521a426eae548b2e7bf11584 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:46.589571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba044465-64ec-4b9a-9c80-4440cebb1244 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:46.686814 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba044465-64ec-4b9a-9c80-4440cebb1244 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:46.718746 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e59aa34-3cc7-4d03-9e83-76fb86d3b2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:46.719346 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59aa34-3cc7-4d03-9e83-76fb86d3b2bb None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:46.719619 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59aa34-3cc7-4d03-9e83-76fb86d3b2bb None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:46.719883 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59aa34-3cc7-4d03-9e83-76fb86d3b2bb None admin] PATH_INFO: `/v3/projects/501bb76cd60a4b6694c18df2bc9311e5` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:46.720958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e59aa34-3cc7-4d03-9e83-76fb86d3b2bb None admin] RBAC: Authorizing `identity:get_project(project_id=501bb76cd60a4b6694c18df2bc9311e5)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:46.722364 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e59aa34-3cc7-4d03-9e83-76fb86d3b2bb None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:46.723966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 553/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:43:46 2022] GET /identity/v3/projects/501bb76cd60a4b6694c18df2bc9311e5 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:46.730423 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59850734-8d30-4fac-ad9f-a010ac0b3105 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:46.826059 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59850734-8d30-4fac-ad9f-a010ac0b3105 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:46.858912 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24593532-a802-40fb-8209-6006826c1db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:46.859480 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24593532-a802-40fb-8209-6006826c1db2 None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:46.859755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24593532-a802-40fb-8209-6006826c1db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:46.860023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24593532-a802-40fb-8209-6006826c1db2 None admin] PATH_INFO: `/v3/projects/501bb76cd60a4b6694c18df2bc9311e5/users/a6931212521a426eae548b2e7bf11584/roles/ff2ed192587843e5b10b5a35949e4c5f` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:46.862106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24593532-a802-40fb-8209-6006826c1db2 None admin] RBAC: Authorizing `identity:create_grant(project_id=501bb76cd60a4b6694c18df2bc9311e5, user_id=a6931212521a426eae548b2e7bf11584, role_id=ff2ed192587843e5b10b5a35949e4c5f)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:46.863533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24593532-a802-40fb-8209-6006826c1db2 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:46.921843 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 570/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Sep 10 23:43:46 2022] PUT /identity/v3/projects/501bb76cd60a4b6694c18df2bc9311e5/users/a6931212521a426eae548b2e7bf11584/roles/ff2ed192587843e5b10b5a35949e4c5f => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:43:49.060129 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28c3b9-a201-4865-9bcb-4789fb5dfb35 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:49.060662 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28c3b9-a201-4865-9bcb-4789fb5dfb35 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:49.060662 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28c3b9-a201-4865-9bcb-4789fb5dfb35 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:49.062006 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 554/1124] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:49.067232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e7a1d-29f7-4d63-90c3-de53905ef168 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:49.067654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e7a1d-29f7-4d63-90c3-de53905ef168 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:49.067776 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e7a1d-29f7-4d63-90c3-de53905ef168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:49.086197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-846e7a1d-29f7-4d63-90c3-de53905ef168 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:49.087439 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-846e7a1d-29f7-4d63-90c3-de53905ef168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:49.110264 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:49.152519 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 571/1125] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:43:49.424800 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e28c3b9-a201-4865-9bcb-4789fb5dfb35 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:49.428684 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:49.522995 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e28c3b9-a201-4865-9bcb-4789fb5dfb35 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:49.555317 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec618a-5b97-4674-ae41-03cde5d6020f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:49.555916 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec618a-5b97-4674-ae41-03cde5d6020f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:49.556217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec618a-5b97-4674-ae41-03cde5d6020f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:49.556484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec618a-5b97-4674-ae41-03cde5d6020f None admin] PATH_INFO: `/v3/roles/ff2ed192587843e5b10b5a35949e4c5f` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:49.558101 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec618a-5b97-4674-ae41-03cde5d6020f None admin] RBAC: Authorizing `identity:get_role(role_id=ff2ed192587843e5b10b5a35949e4c5f)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:49.559475 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec618a-5b97-4674-ae41-03cde5d6020f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:49.560756 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 555/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:43:49 2022] GET /identity/v3/roles/ff2ed192587843e5b10b5a35949e4c5f => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:49.566514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e7a1d-29f7-4d63-90c3-de53905ef168 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:49.663882 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e7a1d-29f7-4d63-90c3-de53905ef168 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:49.697206 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91733b18-88a7-4b62-be79-01eab0242163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:49.697864 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91733b18-88a7-4b62-be79-01eab0242163 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:49.698140 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91733b18-88a7-4b62-be79-01eab0242163 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:49.698405 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91733b18-88a7-4b62-be79-01eab0242163 None admin] PATH_INFO: `/v3/users/a6931212521a426eae548b2e7bf11584` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:49.699543 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91733b18-88a7-4b62-be79-01eab0242163 None admin] RBAC: Authorizing `identity:get_user(user_id=a6931212521a426eae548b2e7bf11584)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:49.700950 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91733b18-88a7-4b62-be79-01eab0242163 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:49.702487 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 572/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:43:49 2022] GET /identity/v3/users/a6931212521a426eae548b2e7bf11584 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:49.708229 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec618a-5b97-4674-ae41-03cde5d6020f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:49.777494 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:49.806295 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec618a-5b97-4674-ae41-03cde5d6020f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:49.838825 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609b2b7-cb2c-491d-8a9b-690297c5cf51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:49.839415 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609b2b7-cb2c-491d-8a9b-690297c5cf51 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:49.839699 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609b2b7-cb2c-491d-8a9b-690297c5cf51 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:49.839962 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609b2b7-cb2c-491d-8a9b-690297c5cf51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:49.904655 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609b2b7-cb2c-491d-8a9b-690297c5cf51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:49.906076 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609b2b7-cb2c-491d-8a9b-690297c5cf51 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:49.972305 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:50.004535 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 556/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:43:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 23:43:50.011256 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91733b18-88a7-4b62-be79-01eab0242163 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:50.038922 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:50.113860 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91733b18-88a7-4b62-be79-01eab0242163 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:50.146345 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d5da3-9b2d-4f1b-9ef2-5b7165b64db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:50.146985 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d5da3-9b2d-4f1b-9ef2-5b7165b64db1 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:50.147271 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d5da3-9b2d-4f1b-9ef2-5b7165b64db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:50.147540 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d5da3-9b2d-4f1b-9ef2-5b7165b64db1 None admin] PATH_INFO: `/v3/projects/501bb76cd60a4b6694c18df2bc9311e5` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:50.148639 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d5da3-9b2d-4f1b-9ef2-5b7165b64db1 None admin] RBAC: Authorizing `identity:get_project(project_id=501bb76cd60a4b6694c18df2bc9311e5)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:50.150848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d5da3-9b2d-4f1b-9ef2-5b7165b64db1 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:50.152506 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 573/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:43:50 2022] GET /identity/v3/projects/501bb76cd60a4b6694c18df2bc9311e5 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:50.158682 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609b2b7-cb2c-491d-8a9b-690297c5cf51 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:50.255613 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609b2b7-cb2c-491d-8a9b-690297c5cf51 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:50.288753 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e90522-7ce2-4fef-81fe-38aa66821cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:50.289333 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e90522-7ce2-4fef-81fe-38aa66821cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:50.289607 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e90522-7ce2-4fef-81fe-38aa66821cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:50.289875 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e90522-7ce2-4fef-81fe-38aa66821cf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:50.290635 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e90522-7ce2-4fef-81fe-38aa66821cf8 None admin] RBAC: Adding query filter params (role.id=ff2ed192587843e5b10b5a35949e4c5f, scope.project.id=501bb76cd60a4b6694c18df2bc9311e5, user.id=a6931212521a426eae548b2e7bf11584) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:50.291181 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e90522-7ce2-4fef-81fe-38aa66821cf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:50.292595 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e90522-7ce2-4fef-81fe-38aa66821cf8 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:50.298200 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 557/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:43:50 2022] GET /identity/v3/role_assignments?user.id=a6931212521a426eae548b2e7bf11584&scope.project.id=501bb76cd60a4b6694c18df2bc9311e5&role.id=ff2ed192587843e5b10b5a35949e4c5f => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:52.431794 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601fef0-f11d-4b66-bd63-6f7e658b199a None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:52.432578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601fef0-f11d-4b66-bd63-6f7e658b199a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:52.432578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601fef0-f11d-4b66-bd63-6f7e658b199a None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:52.433931 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 574/1131] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:52.439027 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb572f-3840-4ab5-9a7f-69e1dd2f9501 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:52.439449 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb572f-3840-4ab5-9a7f-69e1dd2f9501 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:52.439570 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb572f-3840-4ab5-9a7f-69e1dd2f9501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:52.457563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-95fb572f-3840-4ab5-9a7f-69e1dd2f9501 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:52.458758 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-95fb572f-3840-4ab5-9a7f-69e1dd2f9501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:52.486334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:52.522551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 558/1132] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:43:52.787215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601fef0-f11d-4b66-bd63-6f7e658b199a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:52.879488 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:52.886169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601fef0-f11d-4b66-bd63-6f7e658b199a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:52.922532 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5b316-9080-4dc1-9b23-a017d8e58995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:52.923158 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5b316-9080-4dc1-9b23-a017d8e58995 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:52.923430 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5b316-9080-4dc1-9b23-a017d8e58995 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:52.923715 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5b316-9080-4dc1-9b23-a017d8e58995 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:52.924792 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5b316-9080-4dc1-9b23-a017d8e58995 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:52.926192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5b316-9080-4dc1-9b23-a017d8e58995 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:52.927754 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 575/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:43:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:52.934337 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb572f-3840-4ab5-9a7f-69e1dd2f9501 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:53.031033 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb572f-3840-4ab5-9a7f-69e1dd2f9501 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:53.064171 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadccd89-eed9-4799-b723-1fc7948e7539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:53.064706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadccd89-eed9-4799-b723-1fc7948e7539 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:53.064985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadccd89-eed9-4799-b723-1fc7948e7539 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:53.065255 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadccd89-eed9-4799-b723-1fc7948e7539 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:53.065901 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadccd89-eed9-4799-b723-1fc7948e7539 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:53.067357 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadccd89-eed9-4799-b723-1fc7948e7539 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:53.139747 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 559/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Sep 10 23:43:52 2022] POST /identity/v3/projects => generated 311 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:43:55.338848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8090d90-9b52-417c-aabe-0df082b283b5 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:55.339348 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8090d90-9b52-417c-aabe-0df082b283b5 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:55.339457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8090d90-9b52-417c-aabe-0df082b283b5 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:55.340720 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 576/1135] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:55.348493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d08157-8efa-4d0f-a00d-0ce773d171c9 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:55.348918 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d08157-8efa-4d0f-a00d-0ce773d171c9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:55.349022 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d08157-8efa-4d0f-a00d-0ce773d171c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:55.367900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-69d08157-8efa-4d0f-a00d-0ce773d171c9 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:55.369369 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-69d08157-8efa-4d0f-a00d-0ce773d171c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:55.382107 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:55.435912 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 560/1136] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 23:43:55.711778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8090d90-9b52-417c-aabe-0df082b283b5 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:55.808848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8090d90-9b52-417c-aabe-0df082b283b5 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:55.841324 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4ffdc3-b352-4df0-a8bc-9ab8471414ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:55.841928 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ffdc3-b352-4df0-a8bc-9ab8471414ac None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:55.842197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ffdc3-b352-4df0-a8bc-9ab8471414ac None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:55.842459 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ffdc3-b352-4df0-a8bc-9ab8471414ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:55.843556 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4ffdc3-b352-4df0-a8bc-9ab8471414ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:55.844952 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4ffdc3-b352-4df0-a8bc-9ab8471414ac None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:55.846480 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 577/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:43:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:55.852218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d08157-8efa-4d0f-a00d-0ce773d171c9 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:55.947586 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d08157-8efa-4d0f-a00d-0ce773d171c9 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:55.980552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55133b7-1b00-480d-b850-dcbc2e1366db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:55.981083 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55133b7-1b00-480d-b850-dcbc2e1366db None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:55.981360 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55133b7-1b00-480d-b850-dcbc2e1366db None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:55.981629 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55133b7-1b00-480d-b850-dcbc2e1366db None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:55.982279 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55133b7-1b00-480d-b850-dcbc2e1366db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:55.983843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55133b7-1b00-480d-b850-dcbc2e1366db None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:56.081896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 561/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Sep 10 23:43:55 2022] POST /identity/v3/users => generated 286 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:43:58.310206 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899e71b-7afa-45df-8f3f-8be9aea00d78 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:58.310599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899e71b-7afa-45df-8f3f-8be9aea00d78 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:58.310758 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899e71b-7afa-45df-8f3f-8be9aea00d78 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:58.312086 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 578/1139] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:43:58.317246 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe7de5-ec7c-49be-ae6d-b84e4eec4e05 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:58.317625 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe7de5-ec7c-49be-ae6d-b84e4eec4e05 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:58.317786 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe7de5-ec7c-49be-ae6d-b84e4eec4e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:58.325859 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:58.337177 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-98fe7de5-ec7c-49be-ae6d-b84e4eec4e05 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:43:58.346571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-98fe7de5-ec7c-49be-ae6d-b84e4eec4e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:43:58.413014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 562/1140] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:43:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 23:43:58.694124 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899e71b-7afa-45df-8f3f-8be9aea00d78 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:58.724305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:58.792971 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899e71b-7afa-45df-8f3f-8be9aea00d78 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:58.825852 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f12af-7697-403b-b937-2d288a6342f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:58.826457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f12af-7697-403b-b937-2d288a6342f6 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:58.826756 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f12af-7697-403b-b937-2d288a6342f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:58.827024 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f12af-7697-403b-b937-2d288a6342f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:58.836612 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125f12af-7697-403b-b937-2d288a6342f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:43:58.841658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-125f12af-7697-403b-b937-2d288a6342f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:43:58.841658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:43:58.841658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:43:58.841658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:43:58.841658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:43:58.841658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:43:58.841658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:43:58.841658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:43:58.841658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:43:58.841658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:43:58.841658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:43:58.841658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:43:58.841658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:43:58.841658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:43:58.841658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:43:58.841658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:43:58.842822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:43:58.842822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:43:58.842822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:43:58.842822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:43:58.842822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:43:58.842822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:43:58.842822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:43:58.842822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:43:58.842822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:43:58.842822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:43:58.842822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:43:58.842822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:43:58.842822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:43:58.842822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:43:58.842822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:43:58.842822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:43:58.843857 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:43:58.843857 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:43:58.843857 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:43:58.843857 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:43:58.843857 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:43:58.843857 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:43:58.843857 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:43:58.843857 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:43:58.843857 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:43:58.843857 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:43:58.843857 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:43:58.843857 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:43:58.843857 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 579/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:43:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:43:58.848871 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe7de5-ec7c-49be-ae6d-b84e4eec4e05 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:58.944675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe7de5-ec7c-49be-ae6d-b84e4eec4e05 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:58.978281 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fbe04d-6b48-4db6-bac9-c762971c5f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:58.979145 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbe04d-6b48-4db6-bac9-c762971c5f07 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:58.979627 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbe04d-6b48-4db6-bac9-c762971c5f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:58.980117 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbe04d-6b48-4db6-bac9-c762971c5f07 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:58.982677 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fbe04d-6b48-4db6-bac9-c762971c5f07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:58.982677 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fbe04d-6b48-4db6-bac9-c762971c5f07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:58.982677 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fbe04d-6b48-4db6-bac9-c762971c5f07 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:58.984064 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:58.991562 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 563/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:43:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:58.998190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f12af-7697-403b-b937-2d288a6342f6 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:59.135309 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:59.143748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f12af-7697-403b-b937-2d288a6342f6 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:59.176344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b16f49-02b6-4cbe-b667-2e78c21d42c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:59.176961 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b16f49-02b6-4cbe-b667-2e78c21d42c0 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:59.177237 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b16f49-02b6-4cbe-b667-2e78c21d42c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:59.177507 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b16f49-02b6-4cbe-b667-2e78c21d42c0 None admin] PATH_INFO: `/v3/users/b627a83f87a441dbb9a7708d2403c603` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:59.191244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b16f49-02b6-4cbe-b667-2e78c21d42c0 None admin] RBAC: Authorizing `identity:get_user(user_id=b627a83f87a441dbb9a7708d2403c603)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:59.192664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b16f49-02b6-4cbe-b667-2e78c21d42c0 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:59.194236 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 580/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:43:58 2022] GET /identity/v3/users/b627a83f87a441dbb9a7708d2403c603 => generated 286 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:59.202771 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fbe04d-6b48-4db6-bac9-c762971c5f07 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:59.300893 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fbe04d-6b48-4db6-bac9-c762971c5f07 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:59.333536 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2258dcff-ab12-4e36-9de4-5b4af1f8b2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:59.334118 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258dcff-ab12-4e36-9de4-5b4af1f8b2e9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:59.334390 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258dcff-ab12-4e36-9de4-5b4af1f8b2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:59.334681 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258dcff-ab12-4e36-9de4-5b4af1f8b2e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:59.393674 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:59.399828 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2258dcff-ab12-4e36-9de4-5b4af1f8b2e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:59.402071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2258dcff-ab12-4e36-9de4-5b4af1f8b2e9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:59.496407 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 564/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:43:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 23:43:59.502367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b16f49-02b6-4cbe-b667-2e78c21d42c0 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:59.600196 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b16f49-02b6-4cbe-b667-2e78c21d42c0 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:59.632578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18ae64-9c22-4f67-a40d-9efb49f92683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:59.633193 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18ae64-9c22-4f67-a40d-9efb49f92683 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:59.633470 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18ae64-9c22-4f67-a40d-9efb49f92683 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:59.633741 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18ae64-9c22-4f67-a40d-9efb49f92683 None admin] PATH_INFO: `/v3/projects/0b5ed90b491a4eab8b01f9da5bb25399` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:59.635563 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e18ae64-9c22-4f67-a40d-9efb49f92683 None admin] RBAC: Authorizing `identity:get_project(project_id=0b5ed90b491a4eab8b01f9da5bb25399)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:59.636969 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e18ae64-9c22-4f67-a40d-9efb49f92683 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:59.638555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 581/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:43:59 2022] GET /identity/v3/projects/0b5ed90b491a4eab8b01f9da5bb25399 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:43:59.644553 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2258dcff-ab12-4e36-9de4-5b4af1f8b2e9 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:43:59.736035 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:43:59.741633 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2258dcff-ab12-4e36-9de4-5b4af1f8b2e9 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:43:59.775097 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3812b-31d9-4502-b193-c3a9e7130c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:43:59.776198 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3812b-31d9-4502-b193-c3a9e7130c10 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:43:59.776478 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3812b-31d9-4502-b193-c3a9e7130c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:43:59.776748 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3812b-31d9-4502-b193-c3a9e7130c10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:43:59.777463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce3812b-31d9-4502-b193-c3a9e7130c10 None admin] RBAC: Adding query filter params (role.id=9dce48a2b53c482d931dcd064b07744e, scope.project.id=0b5ed90b491a4eab8b01f9da5bb25399, user.id=b627a83f87a441dbb9a7708d2403c603) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:43:59.777766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce3812b-31d9-4502-b193-c3a9e7130c10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:43:59.779210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce3812b-31d9-4502-b193-c3a9e7130c10 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:43:59.784419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 565/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:43:59 2022] GET /identity/v3/role_assignments?user.id=b627a83f87a441dbb9a7708d2403c603&scope.project.id=0b5ed90b491a4eab8b01f9da5bb25399&role.id=9dce48a2b53c482d931dcd064b07744e => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:44:01.910570 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b25ec3-8a06-4e6b-8aa6-f82a1b09f1ca None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:01.911149 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b25ec3-8a06-4e6b-8aa6-f82a1b09f1ca None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:01.911149 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b25ec3-8a06-4e6b-8aa6-f82a1b09f1ca None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:01.912491 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 582/1147] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:44:01.918096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ece98-56ea-45e9-8dc7-2547a73c901b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:01.918527 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ece98-56ea-45e9-8dc7-2547a73c901b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:01.918629 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ece98-56ea-45e9-8dc7-2547a73c901b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:01.937399 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-eb9ece98-56ea-45e9-8dc7-2547a73c901b None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:44:01.938574 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-eb9ece98-56ea-45e9-8dc7-2547a73c901b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:44:02.003875 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 566/1148] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:44:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:44:02.274921 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b25ec3-8a06-4e6b-8aa6-f82a1b09f1ca None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:02.294496 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:02.376642 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b25ec3-8a06-4e6b-8aa6-f82a1b09f1ca None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:02.410171 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225be7e6-4c37-4866-bf13-d10aee768673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:02.410808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225be7e6-4c37-4866-bf13-d10aee768673 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:02.411082 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225be7e6-4c37-4866-bf13-d10aee768673 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:02.411351 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225be7e6-4c37-4866-bf13-d10aee768673 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:02.421810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225be7e6-4c37-4866-bf13-d10aee768673 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:44:02.426486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-225be7e6-4c37-4866-bf13-d10aee768673 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:44:02.426486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:44:02.426486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:44:02.426486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:44:02.426486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:44:02.426486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:44:02.426486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:44:02.426486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:44:02.426486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:44:02.426486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:44:02.426486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:44:02.426486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:44:02.426486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:44:02.426486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:44:02.426486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:44:02.426486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:44:02.427615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:44:02.427615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:44:02.427615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:44:02.427615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:44:02.427615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:44:02.427615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:44:02.427615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:44:02.427615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:44:02.427615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:44:02.427615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:44:02.427615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:44:02.427615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:44:02.427615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:44:02.427615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:44:02.427615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:44:02.427615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:44:02.428640 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:44:02.428640 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:44:02.428640 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:44:02.428640 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:44:02.428640 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:44:02.428640 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:44:02.428640 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:44:02.428640 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:44:02.428640 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:44:02.428640 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:44:02.428640 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:44:02.428640 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:44:02.428640 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 583/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:44:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:44:02.434348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ece98-56ea-45e9-8dc7-2547a73c901b None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:02.521034 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:02.534297 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ece98-56ea-45e9-8dc7-2547a73c901b None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:02.564146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5146a817-7091-47d6-bedc-d544e8936346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:02.564727 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5146a817-7091-47d6-bedc-d544e8936346 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:02.565002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5146a817-7091-47d6-bedc-d544e8936346 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:02.565269 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5146a817-7091-47d6-bedc-d544e8936346 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:02.565918 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5146a817-7091-47d6-bedc-d544e8936346 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:44:02.566216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5146a817-7091-47d6-bedc-d544e8936346 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:02.567596 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5146a817-7091-47d6-bedc-d544e8936346 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:02.575515 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 567/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:44:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:44:02.581385 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225be7e6-4c37-4866-bf13-d10aee768673 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:02.657529 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:02.677755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225be7e6-4c37-4866-bf13-d10aee768673 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:02.710598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d02d4f-d78c-44d6-a479-ed4fa4544ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:02.711239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d02d4f-d78c-44d6-a479-ed4fa4544ccf None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:02.711517 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d02d4f-d78c-44d6-a479-ed4fa4544ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:02.711785 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d02d4f-d78c-44d6-a479-ed4fa4544ccf None admin] PATH_INFO: `/v3/users/b627a83f87a441dbb9a7708d2403c603` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:02.713457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d02d4f-d78c-44d6-a479-ed4fa4544ccf None admin] RBAC: Authorizing `identity:get_user(user_id=b627a83f87a441dbb9a7708d2403c603)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:02.714900 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d02d4f-d78c-44d6-a479-ed4fa4544ccf None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:02.716454 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 584/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:44:02 2022] GET /identity/v3/users/b627a83f87a441dbb9a7708d2403c603 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:44:02.722299 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5146a817-7091-47d6-bedc-d544e8936346 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:02.818403 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5146a817-7091-47d6-bedc-d544e8936346 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:02.850599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe59a3c-82a7-4a4e-b28c-f0266874925e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:02.851233 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe59a3c-82a7-4a4e-b28c-f0266874925e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:02.851509 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe59a3c-82a7-4a4e-b28c-f0266874925e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:02.851784 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe59a3c-82a7-4a4e-b28c-f0266874925e None admin] PATH_INFO: `/v3/projects/0b5ed90b491a4eab8b01f9da5bb25399` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:02.852867 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe59a3c-82a7-4a4e-b28c-f0266874925e None admin] RBAC: Authorizing `identity:get_project(project_id=0b5ed90b491a4eab8b01f9da5bb25399)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:02.854431 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe59a3c-82a7-4a4e-b28c-f0266874925e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:02.856058 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 568/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:44:02 2022] GET /identity/v3/projects/0b5ed90b491a4eab8b01f9da5bb25399 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:44:02.862046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d02d4f-d78c-44d6-a479-ed4fa4544ccf None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:02.958907 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d02d4f-d78c-44d6-a479-ed4fa4544ccf None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:02.991193 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370320a-ad55-409b-a171-07f968c83077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:02.991829 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370320a-ad55-409b-a171-07f968c83077 None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:02.992106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370320a-ad55-409b-a171-07f968c83077 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:02.992375 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370320a-ad55-409b-a171-07f968c83077 None admin] PATH_INFO: `/v3/projects/0b5ed90b491a4eab8b01f9da5bb25399/users/b627a83f87a441dbb9a7708d2403c603/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:02.994801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0370320a-ad55-409b-a171-07f968c83077 None admin] RBAC: Authorizing `identity:create_grant(project_id=0b5ed90b491a4eab8b01f9da5bb25399, user_id=b627a83f87a441dbb9a7708d2403c603, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:02.996211 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0370320a-ad55-409b-a171-07f968c83077 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:03.081422 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 585/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Sep 10 23:44:02 2022] PUT /identity/v3/projects/0b5ed90b491a4eab8b01f9da5bb25399/users/b627a83f87a441dbb9a7708d2403c603/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:44:05.189780 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5fa21-a010-48aa-a749-be5b5f7fa310 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:05.190313 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5fa21-a010-48aa-a749-be5b5f7fa310 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:05.190313 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5fa21-a010-48aa-a749-be5b5f7fa310 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:05.191683 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 569/1154] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:44:05.197532 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d05303-f02c-41dd-bcfd-239a691041aa None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:05.197934 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d05303-f02c-41dd-bcfd-239a691041aa None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:05.198343 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d05303-f02c-41dd-bcfd-239a691041aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:05.210880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:05.216375 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-82d05303-f02c-41dd-bcfd-239a691041aa None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:44:05.217542 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-82d05303-f02c-41dd-bcfd-239a691041aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:44:05.285433 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 586/1155] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:44:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:44:05.560276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5fa21-a010-48aa-a749-be5b5f7fa310 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:05.582074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:05.659219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5fa21-a010-48aa-a749-be5b5f7fa310 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:05.691472 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac685c-1a22-460d-9b3c-c7c80ab26ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:05.692077 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac685c-1a22-460d-9b3c-c7c80ab26ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:05.692350 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac685c-1a22-460d-9b3c-c7c80ab26ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:05.692618 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac685c-1a22-460d-9b3c-c7c80ab26ba2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:05.702713 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac685c-1a22-460d-9b3c-c7c80ab26ba2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:44:05.707359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-8cac685c-1a22-460d-9b3c-c7c80ab26ba2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:44:05.707359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:44:05.707359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:44:05.707359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:44:05.707359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:44:05.707359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:44:05.707359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:44:05.707359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:44:05.707359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:44:05.707359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:44:05.707359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:44:05.707359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:44:05.707359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:44:05.707359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:44:05.707359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:44:05.707359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:44:05.708506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:44:05.708506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:44:05.708506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:44:05.708506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:44:05.708506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:44:05.708506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:44:05.708506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:44:05.708506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:44:05.708506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:44:05.708506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:44:05.708506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:44:05.708506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:44:05.708506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:44:05.708506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:44:05.708506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:44:05.708506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:44:05.709743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:44:05.709743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:44:05.709743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:44:05.709743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:44:05.709743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:44:05.709743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:44:05.709743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:44:05.709743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:44:05.709743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:44:05.709743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:44:05.709743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:44:05.709743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:44:05.709743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 570/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:44:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:44:05.715302 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d05303-f02c-41dd-bcfd-239a691041aa None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:05.815541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d05303-f02c-41dd-bcfd-239a691041aa None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:05.847901 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c823f-e0fd-497a-939c-1c94232ff24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:05.848495 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c823f-e0fd-497a-939c-1c94232ff24f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:05.848773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c823f-e0fd-497a-939c-1c94232ff24f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:05.849045 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c823f-e0fd-497a-939c-1c94232ff24f None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:05.849696 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736c823f-e0fd-497a-939c-1c94232ff24f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:44:05.849998 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736c823f-e0fd-497a-939c-1c94232ff24f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:05.851379 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736c823f-e0fd-497a-939c-1c94232ff24f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:05.853538 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:05.859556 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 587/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:44:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:44:05.865958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac685c-1a22-460d-9b3c-c7c80ab26ba2 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:05.945268 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:05.964607 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac685c-1a22-460d-9b3c-c7c80ab26ba2 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:05.998370 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e7476c-ea73-4ef5-989d-2396b6a88cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:05.998998 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7476c-ea73-4ef5-989d-2396b6a88cad None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:05.999313 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7476c-ea73-4ef5-989d-2396b6a88cad None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:05.999618 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7476c-ea73-4ef5-989d-2396b6a88cad None admin] PATH_INFO: `/v3/users/b627a83f87a441dbb9a7708d2403c603` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:06.000734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e7476c-ea73-4ef5-989d-2396b6a88cad None admin] RBAC: Authorizing `identity:get_user(user_id=b627a83f87a441dbb9a7708d2403c603)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:06.002161 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e7476c-ea73-4ef5-989d-2396b6a88cad None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:06.003745 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 571/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:44:05 2022] GET /identity/v3/users/b627a83f87a441dbb9a7708d2403c603 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:44:06.009523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c823f-e0fd-497a-939c-1c94232ff24f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:06.112160 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c823f-e0fd-497a-939c-1c94232ff24f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:06.143729 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b5448-6aad-44c4-8a71-22b92b5f8f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:06.144321 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b5448-6aad-44c4-8a71-22b92b5f8f3d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:06.144595 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b5448-6aad-44c4-8a71-22b92b5f8f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:06.144863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b5448-6aad-44c4-8a71-22b92b5f8f3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:06.210806 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:06.214888 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b5448-6aad-44c4-8a71-22b92b5f8f3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:06.216323 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b5448-6aad-44c4-8a71-22b92b5f8f3d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:06.313061 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 588/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:44:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 23:44:06.319059 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e7476c-ea73-4ef5-989d-2396b6a88cad None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:06.413164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e7476c-ea73-4ef5-989d-2396b6a88cad None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:06.445995 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9bb088-788d-447d-b579-8b40e4c52890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:06.446592 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9bb088-788d-447d-b579-8b40e4c52890 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:06.446962 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9bb088-788d-447d-b579-8b40e4c52890 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:06.447235 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9bb088-788d-447d-b579-8b40e4c52890 None admin] PATH_INFO: `/v3/projects/0b5ed90b491a4eab8b01f9da5bb25399` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:06.448319 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9bb088-788d-447d-b579-8b40e4c52890 None admin] RBAC: Authorizing `identity:get_project(project_id=0b5ed90b491a4eab8b01f9da5bb25399)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:06.449728 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9bb088-788d-447d-b579-8b40e4c52890 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:06.451339 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 572/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:44:06 2022] GET /identity/v3/projects/0b5ed90b491a4eab8b01f9da5bb25399 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:44:06.457461 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b5448-6aad-44c4-8a71-22b92b5f8f3d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:06.550013 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:06.553450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b5448-6aad-44c4-8a71-22b92b5f8f3d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:06.586145 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a128908-6c0b-49ce-b38d-3d7e2548e2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:06.586750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a128908-6c0b-49ce-b38d-3d7e2548e2c4 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:06.587021 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a128908-6c0b-49ce-b38d-3d7e2548e2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:06.587281 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a128908-6c0b-49ce-b38d-3d7e2548e2c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:06.588483 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a128908-6c0b-49ce-b38d-3d7e2548e2c4 None admin] RBAC: Adding query filter params (role.id=9dce48a2b53c482d931dcd064b07744e, scope.project.id=0b5ed90b491a4eab8b01f9da5bb25399, user.id=b627a83f87a441dbb9a7708d2403c603) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:44:06.588780 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a128908-6c0b-49ce-b38d-3d7e2548e2c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:06.590178 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a128908-6c0b-49ce-b38d-3d7e2548e2c4 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:06.595771 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 589/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:44:06 2022] GET /identity/v3/role_assignments?user.id=b627a83f87a441dbb9a7708d2403c603&scope.project.id=0b5ed90b491a4eab8b01f9da5bb25399&role.id=9dce48a2b53c482d931dcd064b07744e => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:44:08.786504 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37509188-5242-4718-97e3-b2998c720514 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:08.787110 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37509188-5242-4718-97e3-b2998c720514 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:08.787110 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37509188-5242-4718-97e3-b2998c720514 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:08.788495 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 573/1162] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:44:08.793795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b489c5e-865d-4efb-be0f-a9b4ac8135b7 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:08.794189 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b489c5e-865d-4efb-be0f-a9b4ac8135b7 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:08.794335 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b489c5e-865d-4efb-be0f-a9b4ac8135b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:08.812857 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-0b489c5e-865d-4efb-be0f-a9b4ac8135b7 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:44:08.814016 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0b489c5e-865d-4efb-be0f-a9b4ac8135b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:44:08.877151 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 590/1163] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:44:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:44:09.141004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37509188-5242-4718-97e3-b2998c720514 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:09.159120 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:09.239716 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37509188-5242-4718-97e3-b2998c720514 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:09.272323 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95e164-e13f-43f6-af0b-9e74aecd4e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:09.272898 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95e164-e13f-43f6-af0b-9e74aecd4e14 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:09.273170 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95e164-e13f-43f6-af0b-9e74aecd4e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:09.273435 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95e164-e13f-43f6-af0b-9e74aecd4e14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:09.341045 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab95e164-e13f-43f6-af0b-9e74aecd4e14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:09.342493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab95e164-e13f-43f6-af0b-9e74aecd4e14 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:09.359045 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:09.438201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 574/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:44:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 23:44:09.444138 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b489c5e-865d-4efb-be0f-a9b4ac8135b7 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:09.531251 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:09.539473 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b489c5e-865d-4efb-be0f-a9b4ac8135b7 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:09.573559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a057ad9-abf1-4389-a25b-9170b6383675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:09.574169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a057ad9-abf1-4389-a25b-9170b6383675 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:09.574447 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a057ad9-abf1-4389-a25b-9170b6383675 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:09.574748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a057ad9-abf1-4389-a25b-9170b6383675 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:09.581201 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a057ad9-abf1-4389-a25b-9170b6383675 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:09.582628 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a057ad9-abf1-4389-a25b-9170b6383675 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:09.594141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-1a057ad9-abf1-4389-a25b-9170b6383675 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 10 23:44:09.594141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:44:09.594141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 10 23:44:09.594141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 10 23:44:09.594141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:44:09.594141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:44:09.594141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:44:09.594141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:44:09.594141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 10 23:44:09.594141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:44:09.594141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:44:09.594141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:44:09.594141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:44:09.594141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:44:09.594141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:44:09.594141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:44:09.594141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:44:09.595219 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:44:09.595219 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:44:09.595219 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:44:09.595219 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:44:09.595219 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:44:09.595219 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:44:09.595219 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:44:09.595219 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:44:09.595219 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:44:09.595219 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:44:09.595219 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:44:09.595219 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:44:09.595219 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:44:09.595219 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:44:09.595219 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:44:09.595219 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:44:09.596264 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:44:09.596264 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:44:09.596264 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:44:09.596264 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:44:09.596264 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:44:09.596264 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:44:09.596264 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:44:09.596264 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:44:09.596264 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:44:09.596264 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:44:09.596264 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 10 23:44:09.596264 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:44:09.596264 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 10 23:44:09.596264 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:44:09.596264 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 591/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Sep 10 23:44:09 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:44:09.601288 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95e164-e13f-43f6-af0b-9e74aecd4e14 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:09.698262 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95e164-e13f-43f6-af0b-9e74aecd4e14 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:09.714148 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:09.731177 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3078c48-a672-44bf-aae9-c0223e5dccf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:09.731767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3078c48-a672-44bf-aae9-c0223e5dccf0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:09.732046 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3078c48-a672-44bf-aae9-c0223e5dccf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:09.732316 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3078c48-a672-44bf-aae9-c0223e5dccf0 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:09.732956 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3078c48-a672-44bf-aae9-c0223e5dccf0 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:44:09.733255 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3078c48-a672-44bf-aae9-c0223e5dccf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:09.734619 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3078c48-a672-44bf-aae9-c0223e5dccf0 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:09.741499 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 575/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Sat Sep 10 23:44:09 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:44:11.846526 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b8f11-36c5-4438-b7cb-26d08fab3856 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:11.847106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b8f11-36c5-4438-b7cb-26d08fab3856 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:11.847106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b8f11-36c5-4438-b7cb-26d08fab3856 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:11.848406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 592/1167] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:44:11.853607 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd312e-c140-4a1a-a1bf-988120b560be None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:11.854003 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd312e-c140-4a1a-a1bf-988120b560be None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:11.854146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd312e-c140-4a1a-a1bf-988120b560be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:11.872416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b0cd312e-c140-4a1a-a1bf-988120b560be None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:44:11.873591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b0cd312e-c140-4a1a-a1bf-988120b560be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:44:11.937594 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 576/1168] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:44:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:44:12.214877 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b8f11-36c5-4438-b7cb-26d08fab3856 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:12.313857 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b8f11-36c5-4438-b7cb-26d08fab3856 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:12.351486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f883157-552d-4013-bebd-b0f4ad49afda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:12.352087 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f883157-552d-4013-bebd-b0f4ad49afda None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:12.352361 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f883157-552d-4013-bebd-b0f4ad49afda None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:12.352632 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f883157-552d-4013-bebd-b0f4ad49afda None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:12.353639 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f883157-552d-4013-bebd-b0f4ad49afda None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:12.355038 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f883157-552d-4013-bebd-b0f4ad49afda None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:12.436332 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 593/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Sat Sep 10 23:44:12 2022] POST /identity/v3/domains => generated 267 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:44:14.622780 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb1d0b-3895-4c3f-abac-b2dc7d2a6d7c None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:14.623292 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb1d0b-3895-4c3f-abac-b2dc7d2a6d7c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:14.623292 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb1d0b-3895-4c3f-abac-b2dc7d2a6d7c None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:14.624618 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 577/1170] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:44:14.629800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bace4c-a52e-48ef-a367-4a93cced79b3 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:14.630178 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bace4c-a52e-48ef-a367-4a93cced79b3 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:14.630337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bace4c-a52e-48ef-a367-4a93cced79b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:14.648680 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-41bace4c-a52e-48ef-a367-4a93cced79b3 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:44:14.650495 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-41bace4c-a52e-48ef-a367-4a93cced79b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:44:14.657709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:14.716480 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 594/1171] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:44:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:44:14.986057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb1d0b-3895-4c3f-abac-b2dc7d2a6d7c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:15.087894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb1d0b-3895-4c3f-abac-b2dc7d2a6d7c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:15.098664 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:15.121761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765e59b-350c-4a29-a9c6-8896930ffae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:15.122369 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765e59b-350c-4a29-a9c6-8896930ffae1 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:15.122643 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765e59b-350c-4a29-a9c6-8896930ffae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:15.122942 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765e59b-350c-4a29-a9c6-8896930ffae1 None admin] PATH_INFO: `/v3/domains/dbf1ac79b92e45c6b3f14d664fe11b14` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:15.133048 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765e59b-350c-4a29-a9c6-8896930ffae1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=dbf1ac79b92e45c6b3f14d664fe11b14)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:15.135647 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765e59b-350c-4a29-a9c6-8896930ffae1 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:15.137208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 578/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Sat Sep 10 23:44:14 2022] GET /identity/v3/domains/dbf1ac79b92e45c6b3f14d664fe11b14 => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:44:15.143049 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bace4c-a52e-48ef-a367-4a93cced79b3 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:15.239605 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bace4c-a52e-48ef-a367-4a93cced79b3 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:15.271860 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeba209-b2cf-45f1-9f12-cc58d06339c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:15.272444 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeba209-b2cf-45f1-9f12-cc58d06339c5 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:15.272719 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeba209-b2cf-45f1-9f12-cc58d06339c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:15.272987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeba209-b2cf-45f1-9f12-cc58d06339c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:15.273618 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeba209-b2cf-45f1-9f12-cc58d06339c5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:15.275642 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeba209-b2cf-45f1-9f12-cc58d06339c5 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:15.289111 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:15.345362 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 595/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Sep 10 23:44:15 2022] POST /identity/v3/projects => generated 361 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:44:17.517832 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c524ca5-2ecf-45d0-a221-57868ac9e7ac None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:17.518377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c524ca5-2ecf-45d0-a221-57868ac9e7ac None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:17.518377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c524ca5-2ecf-45d0-a221-57868ac9e7ac None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:17.519764 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 579/1174] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:44:17.525639 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb69f2d-1b9e-4c63-a250-d5b89634c347 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:17.526067 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb69f2d-1b9e-4c63-a250-d5b89634c347 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:17.526160 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb69f2d-1b9e-4c63-a250-d5b89634c347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:17.543737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-bbb69f2d-1b9e-4c63-a250-d5b89634c347 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:44:17.544894 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-bbb69f2d-1b9e-4c63-a250-d5b89634c347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:44:17.608146 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 596/1175] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:44:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:44:17.873362 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c524ca5-2ecf-45d0-a221-57868ac9e7ac None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:17.975860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c524ca5-2ecf-45d0-a221-57868ac9e7ac None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:18.008601 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889b47a-3843-478a-a80d-a5f44cfe0e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:18.009973 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889b47a-3843-478a-a80d-a5f44cfe0e6d None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:18.010341 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889b47a-3843-478a-a80d-a5f44cfe0e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:18.010618 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889b47a-3843-478a-a80d-a5f44cfe0e6d None admin] PATH_INFO: `/v3/domains/dbf1ac79b92e45c6b3f14d664fe11b14` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:18.011782 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6889b47a-3843-478a-a80d-a5f44cfe0e6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=dbf1ac79b92e45c6b3f14d664fe11b14)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:18.013245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6889b47a-3843-478a-a80d-a5f44cfe0e6d None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:18.014823 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 580/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Sat Sep 10 23:44:17 2022] GET /identity/v3/domains/dbf1ac79b92e45c6b3f14d664fe11b14 => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:44:18.020522 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb69f2d-1b9e-4c63-a250-d5b89634c347 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:18.113412 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb69f2d-1b9e-4c63-a250-d5b89634c347 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:18.136270 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:18.146060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5be64f-de32-4f9d-b553-c7801ce8b407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:18.146647 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5be64f-de32-4f9d-b553-c7801ce8b407 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:18.146945 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5be64f-de32-4f9d-b553-c7801ce8b407 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:18.147208 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5be64f-de32-4f9d-b553-c7801ce8b407 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:18.147837 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5be64f-de32-4f9d-b553-c7801ce8b407 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:18.149230 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5be64f-de32-4f9d-b553-c7801ce8b407 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:18.240381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 597/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Sep 10 23:44:18 2022] POST /identity/v3/users => generated 311 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:44:20.472268 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678acdb-d8c9-4284-a4ad-bb7df2d690aa None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:20.472790 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678acdb-d8c9-4284-a4ad-bb7df2d690aa None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:20.472790 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678acdb-d8c9-4284-a4ad-bb7df2d690aa None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:20.474144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 581/1178] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:44:20.480029 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d95a1-288d-4846-a117-b580cb890a3d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:20.480438 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d95a1-288d-4846-a117-b580cb890a3d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:20.480558 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d95a1-288d-4846-a117-b580cb890a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:20.498960 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-979d95a1-288d-4846-a117-b580cb890a3d None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:44:20.500120 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-979d95a1-288d-4846-a117-b580cb890a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:44:20.563423 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 598/1179] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:44:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:44:20.833561 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678acdb-d8c9-4284-a4ad-bb7df2d690aa None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:20.874888 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:20.934407 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678acdb-d8c9-4284-a4ad-bb7df2d690aa None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:20.967314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f3337-9ee4-4bce-9cae-50c384d9a42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:20.967922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f3337-9ee4-4bce-9cae-50c384d9a42f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:20.968199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f3337-9ee4-4bce-9cae-50c384d9a42f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:20.968467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f3337-9ee4-4bce-9cae-50c384d9a42f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:20.978938 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667f3337-9ee4-4bce-9cae-50c384d9a42f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:44:20.984014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-667f3337-9ee4-4bce-9cae-50c384d9a42f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:44:20.984014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:44:20.984014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:44:20.984014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:44:20.984014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:44:20.984014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:44:20.984014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:44:20.984014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:44:20.984014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:44:20.984014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:44:20.984014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:44:20.984014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:44:20.984014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:44:20.984014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:44:20.984014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:44:20.984014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:44:20.985190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:44:20.985190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:44:20.985190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:44:20.985190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:44:20.985190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:44:20.985190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:44:20.985190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:44:20.985190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:44:20.985190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:44:20.985190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:44:20.985190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:44:20.985190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:44:20.985190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:44:20.985190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:44:20.985190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:44:20.985190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:44:20.986518 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:44:20.986518 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:44:20.986518 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:44:20.986518 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:44:20.986518 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:44:20.986518 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:44:20.986518 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:44:20.986518 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:44:20.986518 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:44:20.986518 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:44:20.986518 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:44:20.986518 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:44:20.986518 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 582/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:44:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:44:20.991502 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d95a1-288d-4846-a117-b580cb890a3d None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:21.089634 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d95a1-288d-4846-a117-b580cb890a3d None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:21.099874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:21.123086 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6dce53-d100-4265-8c6c-06ba6a55934a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:21.124323 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dce53-d100-4265-8c6c-06ba6a55934a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:21.124323 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dce53-d100-4265-8c6c-06ba6a55934a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:21.124323 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dce53-d100-4265-8c6c-06ba6a55934a None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:21.124825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6dce53-d100-4265-8c6c-06ba6a55934a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:44:21.125118 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6dce53-d100-4265-8c6c-06ba6a55934a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:21.126500 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6dce53-d100-4265-8c6c-06ba6a55934a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:21.134705 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 599/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:44:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:44:21.140655 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f3337-9ee4-4bce-9cae-50c384d9a42f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:21.234449 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:21.238276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f3337-9ee4-4bce-9cae-50c384d9a42f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:21.272546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db417aff-5485-4bb6-80b8-de875c8f0292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:21.273149 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db417aff-5485-4bb6-80b8-de875c8f0292 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:21.273425 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db417aff-5485-4bb6-80b8-de875c8f0292 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:21.273695 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db417aff-5485-4bb6-80b8-de875c8f0292 None admin] PATH_INFO: `/v3/users/9824ec8307c04375bb45a49564ba32bb` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:21.287182 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db417aff-5485-4bb6-80b8-de875c8f0292 None admin] RBAC: Authorizing `identity:get_user(user_id=9824ec8307c04375bb45a49564ba32bb)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:21.288622 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db417aff-5485-4bb6-80b8-de875c8f0292 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:21.290242 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 583/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:44:21 2022] GET /identity/v3/users/9824ec8307c04375bb45a49564ba32bb => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:44:21.296101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6dce53-d100-4265-8c6c-06ba6a55934a None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:21.398478 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6dce53-d100-4265-8c6c-06ba6a55934a None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:21.430347 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6e001-8962-46b8-96fc-2ed8efd0d8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:21.431073 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6e001-8962-46b8-96fc-2ed8efd0d8fc None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:21.431346 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6e001-8962-46b8-96fc-2ed8efd0d8fc None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:21.431713 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6e001-8962-46b8-96fc-2ed8efd0d8fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:21.470822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:21.495536 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6e001-8962-46b8-96fc-2ed8efd0d8fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:21.496962 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6e001-8962-46b8-96fc-2ed8efd0d8fc None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:21.590476 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 600/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:44:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 23:44:21.596743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db417aff-5485-4bb6-80b8-de875c8f0292 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:21.693652 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db417aff-5485-4bb6-80b8-de875c8f0292 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:21.726922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8daa280-16a2-4810-90f3-1014c180a5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:21.727537 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8daa280-16a2-4810-90f3-1014c180a5a1 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:21.727833 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8daa280-16a2-4810-90f3-1014c180a5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:21.728101 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8daa280-16a2-4810-90f3-1014c180a5a1 None admin] PATH_INFO: `/v3/projects/72bffc58f6f44dbcb987edadd2fdf897` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:21.729387 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8daa280-16a2-4810-90f3-1014c180a5a1 None admin] RBAC: Authorizing `identity:get_project(project_id=72bffc58f6f44dbcb987edadd2fdf897)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:21.730839 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8daa280-16a2-4810-90f3-1014c180a5a1 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:21.732448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 584/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:44:21 2022] GET /identity/v3/projects/72bffc58f6f44dbcb987edadd2fdf897 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:44:21.738653 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6e001-8962-46b8-96fc-2ed8efd0d8fc None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:21.804534 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:21.832748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6e001-8962-46b8-96fc-2ed8efd0d8fc None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:21.864770 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9220ecf-b0d6-4bb0-ad4c-855f41ffe79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:21.865355 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9220ecf-b0d6-4bb0-ad4c-855f41ffe79d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:21.865629 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9220ecf-b0d6-4bb0-ad4c-855f41ffe79d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:21.865897 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9220ecf-b0d6-4bb0-ad4c-855f41ffe79d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:21.866603 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9220ecf-b0d6-4bb0-ad4c-855f41ffe79d None admin] RBAC: Adding query filter params (role.id=9dce48a2b53c482d931dcd064b07744e, scope.project.id=72bffc58f6f44dbcb987edadd2fdf897, user.id=9824ec8307c04375bb45a49564ba32bb) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:44:21.866932 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9220ecf-b0d6-4bb0-ad4c-855f41ffe79d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:21.868843 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9220ecf-b0d6-4bb0-ad4c-855f41ffe79d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:21.873962 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 601/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:44:21 2022] GET /identity/v3/role_assignments?user.id=9824ec8307c04375bb45a49564ba32bb&scope.project.id=72bffc58f6f44dbcb987edadd2fdf897&role.id=9dce48a2b53c482d931dcd064b07744e => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:44:23.980151 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011650b4-c420-4155-a67f-716ff32bf9f4 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:23.980720 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011650b4-c420-4155-a67f-716ff32bf9f4 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:23.980720 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011650b4-c420-4155-a67f-716ff32bf9f4 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:23.982031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 585/1186] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:44:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:44:23.987239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c1ea6-990c-4adf-ac30-b05e92458cc2 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:23.987659 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c1ea6-990c-4adf-ac30-b05e92458cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:23.987768 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c1ea6-990c-4adf-ac30-b05e92458cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:24.006019 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-5f0c1ea6-990c-4adf-ac30-b05e92458cc2 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:44:24.007223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5f0c1ea6-990c-4adf-ac30-b05e92458cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:44:24.071626 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 602/1187] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:44:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:44:24.341935 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011650b4-c420-4155-a67f-716ff32bf9f4 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:24.367081 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:24.442768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011650b4-c420-4155-a67f-716ff32bf9f4 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:24.476444 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3600386-4557-407e-8bd7-a23185b84e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:24.477044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3600386-4557-407e-8bd7-a23185b84e17 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:24.477320 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3600386-4557-407e-8bd7-a23185b84e17 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:24.477687 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3600386-4557-407e-8bd7-a23185b84e17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:24.487596 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3600386-4557-407e-8bd7-a23185b84e17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:44:24.492276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-d3600386-4557-407e-8bd7-a23185b84e17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:44:24.492276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:44:24.492276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:44:24.492276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:44:24.492276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:44:24.492276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:44:24.492276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:44:24.492276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:44:24.492276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:44:24.492276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:44:24.492276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:44:24.492276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:44:24.492276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:44:24.492276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:44:24.492276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:44:24.492276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:44:24.493377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:44:24.493377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:44:24.493377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:44:24.493377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:44:24.493377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:44:24.493377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:44:24.493377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:44:24.493377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:44:24.493377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:44:24.493377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:44:24.493377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:44:24.493377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:44:24.493377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:44:24.493377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:44:24.493377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:44:24.493377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:44:24.494516 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:44:24.494516 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:44:24.494516 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:44:24.494516 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:44:24.494516 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:44:24.494516 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:44:24.494516 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:44:24.494516 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:44:24.494516 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:44:24.494516 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:44:24.494516 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:44:24.494516 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:44:24.494516 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 586/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:44:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:44:24.499604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c1ea6-990c-4adf-ac30-b05e92458cc2 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:24.563373 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:24.596209 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c1ea6-990c-4adf-ac30-b05e92458cc2 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:24.628165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c03aaf-7e32-433b-b87d-35aa8d008d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:24.628752 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c03aaf-7e32-433b-b87d-35aa8d008d0a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:24.629024 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c03aaf-7e32-433b-b87d-35aa8d008d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:24.629290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c03aaf-7e32-433b-b87d-35aa8d008d0a None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:24.629932 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c03aaf-7e32-433b-b87d-35aa8d008d0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:44:24.630252 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c03aaf-7e32-433b-b87d-35aa8d008d0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:24.631625 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c03aaf-7e32-433b-b87d-35aa8d008d0a None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:24.639931 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 603/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:44:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:44:24.645643 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3600386-4557-407e-8bd7-a23185b84e17 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:24.726578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:24.741510 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3600386-4557-407e-8bd7-a23185b84e17 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:24.775332 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cbd101-f7db-445d-9711-c1c1145bf98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:24.775944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbd101-f7db-445d-9711-c1c1145bf98c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:24.776226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbd101-f7db-445d-9711-c1c1145bf98c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:24.776499 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbd101-f7db-445d-9711-c1c1145bf98c None admin] PATH_INFO: `/v3/users/9824ec8307c04375bb45a49564ba32bb` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:24.777595 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cbd101-f7db-445d-9711-c1c1145bf98c None admin] RBAC: Authorizing `identity:get_user(user_id=9824ec8307c04375bb45a49564ba32bb)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:24.779086 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cbd101-f7db-445d-9711-c1c1145bf98c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:24.780643 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 587/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:44:24 2022] GET /identity/v3/users/9824ec8307c04375bb45a49564ba32bb => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:44:24.786453 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c03aaf-7e32-433b-b87d-35aa8d008d0a None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:24.881937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c03aaf-7e32-433b-b87d-35aa8d008d0a None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:24.907044 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:24.914233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c65c536-945c-481f-ac51-53802a287289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:24.915518 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65c536-945c-481f-ac51-53802a287289 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:24.915816 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65c536-945c-481f-ac51-53802a287289 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:24.916084 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65c536-945c-481f-ac51-53802a287289 None admin] PATH_INFO: `/v3/projects/72bffc58f6f44dbcb987edadd2fdf897` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:24.917184 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c65c536-945c-481f-ac51-53802a287289 None admin] RBAC: Authorizing `identity:get_project(project_id=72bffc58f6f44dbcb987edadd2fdf897)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:24.918595 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c65c536-945c-481f-ac51-53802a287289 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:24.920295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 604/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:44:24 2022] GET /identity/v3/projects/72bffc58f6f44dbcb987edadd2fdf897 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:44:24.926109 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cbd101-f7db-445d-9711-c1c1145bf98c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:25.023505 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cbd101-f7db-445d-9711-c1c1145bf98c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:25.057108 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59557e2-bb6c-48cf-8a29-ca1685f90cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:25.058001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59557e2-bb6c-48cf-8a29-ca1685f90cea None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:25.058282 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59557e2-bb6c-48cf-8a29-ca1685f90cea None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:25.058557 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59557e2-bb6c-48cf-8a29-ca1685f90cea None admin] PATH_INFO: `/v3/projects/72bffc58f6f44dbcb987edadd2fdf897/users/9824ec8307c04375bb45a49564ba32bb/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:25.060457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59557e2-bb6c-48cf-8a29-ca1685f90cea None admin] RBAC: Authorizing `identity:create_grant(project_id=72bffc58f6f44dbcb987edadd2fdf897, user_id=9824ec8307c04375bb45a49564ba32bb, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:25.061880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59557e2-bb6c-48cf-8a29-ca1685f90cea None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:25.114082 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 588/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Sep 10 23:44:24 2022] PUT /identity/v3/projects/72bffc58f6f44dbcb987edadd2fdf897/users/9824ec8307c04375bb45a49564ba32bb/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:44:27.252214 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a09f49-694e-41b2-b1f6-b961992442d4 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:27.252750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a09f49-694e-41b2-b1f6-b961992442d4 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:27.252750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a09f49-694e-41b2-b1f6-b961992442d4 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:27.254060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 605/1193] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:44:27.259917 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b21a3-d5fd-4a61-8615-c901803cb2a6 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:27.260343 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b21a3-d5fd-4a61-8615-c901803cb2a6 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:27.260469 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b21a3-d5fd-4a61-8615-c901803cb2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:27.278705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-781b21a3-d5fd-4a61-8615-c901803cb2a6 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:44:27.279895 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-781b21a3-d5fd-4a61-8615-c901803cb2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:44:27.287123 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:27.346265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 589/1194] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:44:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:44:27.627328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a09f49-694e-41b2-b1f6-b961992442d4 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:27.736159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a09f49-694e-41b2-b1f6-b961992442d4 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:27.767645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c645fb-3b8f-477b-b67f-4aec8c14fa2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:27.768260 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c645fb-3b8f-477b-b67f-4aec8c14fa2a None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:27.768535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c645fb-3b8f-477b-b67f-4aec8c14fa2a None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:27.768801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c645fb-3b8f-477b-b67f-4aec8c14fa2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:27.779446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c645fb-3b8f-477b-b67f-4aec8c14fa2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:44:27.784145 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-e5c645fb-3b8f-477b-b67f-4aec8c14fa2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:44:27.784145 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:44:27.784145 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:44:27.784145 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:44:27.784145 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:44:27.784145 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:44:27.784145 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:44:27.784145 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:44:27.784145 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:44:27.784145 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:44:27.784145 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:44:27.784145 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:44:27.784145 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:44:27.784145 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:44:27.784145 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:44:27.784145 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:44:27.785239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:44:27.785239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:44:27.785239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:44:27.785239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:44:27.785239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:44:27.785239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:44:27.785239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:44:27.785239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:44:27.785239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:44:27.785239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:44:27.785239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:44:27.785239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:44:27.785239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:44:27.785239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:44:27.785239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:44:27.785239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:44:27.786197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:44:27.786197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:44:27.786197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:44:27.786197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:44:27.786197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:44:27.786197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:44:27.786197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:44:27.786197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:44:27.786197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:44:27.786197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:44:27.786197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:44:27.786197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:44:27.786197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 606/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:44:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:44:27.792263 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b21a3-d5fd-4a61-8615-c901803cb2a6 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:27.886987 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b21a3-d5fd-4a61-8615-c901803cb2a6 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:27.919971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79647d5b-16bf-47c7-83db-e1576cebbfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:27.920550 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79647d5b-16bf-47c7-83db-e1576cebbfaf None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:27.920826 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79647d5b-16bf-47c7-83db-e1576cebbfaf None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:27.921097 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79647d5b-16bf-47c7-83db-e1576cebbfaf None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:27.921750 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79647d5b-16bf-47c7-83db-e1576cebbfaf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:44:27.922051 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79647d5b-16bf-47c7-83db-e1576cebbfaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:27.923437 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79647d5b-16bf-47c7-83db-e1576cebbfaf None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:27.931283 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 590/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:44:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:44:27.937282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c645fb-3b8f-477b-b67f-4aec8c14fa2a None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:27.992438 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:28.034984 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c645fb-3b8f-477b-b67f-4aec8c14fa2a None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:28.067562 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8543d8-d1b5-4df7-ae98-05abaf849386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:28.068170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8543d8-d1b5-4df7-ae98-05abaf849386 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:28.068442 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8543d8-d1b5-4df7-ae98-05abaf849386 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:28.068707 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8543d8-d1b5-4df7-ae98-05abaf849386 None admin] PATH_INFO: `/v3/users/9824ec8307c04375bb45a49564ba32bb` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:28.069796 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8543d8-d1b5-4df7-ae98-05abaf849386 None admin] RBAC: Authorizing `identity:get_user(user_id=9824ec8307c04375bb45a49564ba32bb)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:28.071762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8543d8-d1b5-4df7-ae98-05abaf849386 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:28.073325 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 607/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Sat Sep 10 23:44:27 2022] GET /identity/v3/users/9824ec8307c04375bb45a49564ba32bb => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:44:28.079176 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79647d5b-16bf-47c7-83db-e1576cebbfaf None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:28.082897 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:28.176624 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79647d5b-16bf-47c7-83db-e1576cebbfaf None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:28.208732 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe116ec-acc2-4002-9ac6-dfab53bedcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:28.209312 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe116ec-acc2-4002-9ac6-dfab53bedcc1 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:28.209582 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe116ec-acc2-4002-9ac6-dfab53bedcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:28.209843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe116ec-acc2-4002-9ac6-dfab53bedcc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:28.275082 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe116ec-acc2-4002-9ac6-dfab53bedcc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:28.276434 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe116ec-acc2-4002-9ac6-dfab53bedcc1 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:28.278950 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:28.372789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 591/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:44:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 23:44:28.380786 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8543d8-d1b5-4df7-ae98-05abaf849386 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:28.481005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8543d8-d1b5-4df7-ae98-05abaf849386 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:28.503697 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:28.514818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe3c252-0644-4171-971a-1acc322eba69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:28.515437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3c252-0644-4171-971a-1acc322eba69 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:28.515709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3c252-0644-4171-971a-1acc322eba69 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:28.515966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3c252-0644-4171-971a-1acc322eba69 None admin] PATH_INFO: `/v3/projects/72bffc58f6f44dbcb987edadd2fdf897` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:28.517052 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe3c252-0644-4171-971a-1acc322eba69 None admin] RBAC: Authorizing `identity:get_project(project_id=72bffc58f6f44dbcb987edadd2fdf897)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:28.518460 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe3c252-0644-4171-971a-1acc322eba69 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:28.520179 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 608/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Sat Sep 10 23:44:28 2022] GET /identity/v3/projects/72bffc58f6f44dbcb987edadd2fdf897 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:44:28.526482 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe116ec-acc2-4002-9ac6-dfab53bedcc1 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:44:28.620505 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe116ec-acc2-4002-9ac6-dfab53bedcc1 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:44:28.622760 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:28.654727 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6240e-360e-4145-a157-3501f48dbe59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:44:28.655329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6240e-360e-4145-a157-3501f48dbe59 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:44:28.655609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6240e-360e-4145-a157-3501f48dbe59 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:44:28.655876 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6240e-360e-4145-a157-3501f48dbe59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:44:28.656590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e6240e-360e-4145-a157-3501f48dbe59 None admin] RBAC: Adding query filter params (role.id=9dce48a2b53c482d931dcd064b07744e, scope.project.id=72bffc58f6f44dbcb987edadd2fdf897, user.id=9824ec8307c04375bb45a49564ba32bb) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:44:28.656889 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e6240e-360e-4145-a157-3501f48dbe59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:44:28.658306 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e6240e-360e-4145-a157-3501f48dbe59 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:44:28.664012 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 592/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:44:28 2022] GET /identity/v3/role_assignments?user.id=9824ec8307c04375bb45a49564ba32bb&scope.project.id=72bffc58f6f44dbcb987edadd2fdf897&role.id=9dce48a2b53c482d931dcd064b07744e => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:44:38.531612 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:44:38.661556 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=16 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:39.053690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfa040-c0b3-4bb6-b908-277ff0874c23 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:39.054289 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfa040-c0b3-4bb6-b908-277ff0874c23 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:39.054289 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfa040-c0b3-4bb6-b908-277ff0874c23 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:39.055601 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 609/1201] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:45:39.061043 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159be27b-f88f-42fc-80c1-322237179b32 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:39.061473 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159be27b-f88f-42fc-80c1-322237179b32 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:39.061579 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159be27b-f88f-42fc-80c1-322237179b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:39.080110 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-159be27b-f88f-42fc-80c1-322237179b32 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:45:39.081282 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-159be27b-f88f-42fc-80c1-322237179b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:45:39.148303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 593/1202] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:45:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:45:39.417652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfa040-c0b3-4bb6-b908-277ff0874c23 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:39.516183 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfa040-c0b3-4bb6-b908-277ff0874c23 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:39.550282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac4b77-b88b-4b1b-8da1-78795782c392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:39.550932 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac4b77-b88b-4b1b-8da1-78795782c392 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:39.551206 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac4b77-b88b-4b1b-8da1-78795782c392 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:39.551499 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac4b77-b88b-4b1b-8da1-78795782c392 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:39.564514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac4b77-b88b-4b1b-8da1-78795782c392 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:39.564514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac4b77-b88b-4b1b-8da1-78795782c392 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:39.577182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-40ac4b77-b88b-4b1b-8da1-78795782c392 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:45:39.577182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:45:39.577182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:45:39.577182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:45:39.577182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:45:39.577182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:45:39.577182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:45:39.577182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:45:39.577182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:45:39.577182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:45:39.577182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:45:39.577182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:45:39.577182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:45:39.577182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:45:39.577182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:45:39.577182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:45:39.578357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:45:39.578357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:45:39.578357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:45:39.578357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:45:39.578357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:45:39.578357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:45:39.578357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:45:39.578357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:45:39.578357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:45:39.578357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:45:39.578357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:45:39.578357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:45:39.578357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:45:39.578357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:45:39.578357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:45:39.578357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:45:39.578357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:45:39.579593 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:45:39.579593 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:45:39.579593 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:45:39.579593 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 610/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:45:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:45:39.584488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159be27b-f88f-42fc-80c1-322237179b32 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:39.683385 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159be27b-f88f-42fc-80c1-322237179b32 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:39.712126 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:39.718283 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e907ca-873e-417f-803d-cbe6a9dfcd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:39.718974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e907ca-873e-417f-803d-cbe6a9dfcd71 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:39.719250 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e907ca-873e-417f-803d-cbe6a9dfcd71 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:39.719516 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e907ca-873e-417f-803d-cbe6a9dfcd71 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:39.720140 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e907ca-873e-417f-803d-cbe6a9dfcd71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:45:39.720435 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e907ca-873e-417f-803d-cbe6a9dfcd71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:39.721788 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e907ca-873e-417f-803d-cbe6a9dfcd71 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:39.733103 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 594/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:45:39.739072 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac4b77-b88b-4b1b-8da1-78795782c392 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:39.790256 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:39.838129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac4b77-b88b-4b1b-8da1-78795782c392 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:39.873245 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7195f4a5-f353-48db-903f-ff754e5b93d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:39.874218 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7195f4a5-f353-48db-903f-ff754e5b93d9 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:39.874745 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7195f4a5-f353-48db-903f-ff754e5b93d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:39.875223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7195f4a5-f353-48db-903f-ff754e5b93d9 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:39.885563 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7195f4a5-f353-48db-903f-ff754e5b93d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:39.885563 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7195f4a5-f353-48db-903f-ff754e5b93d9 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:39.947179 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 611/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Sat Sep 10 23:45:39 2022] POST /identity/v3/users => generated 251 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:45:42.144279 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddef38-f739-4974-949b-3bb24d0ace25 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:42.144839 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddef38-f739-4974-949b-3bb24d0ace25 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:42.144839 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddef38-f739-4974-949b-3bb24d0ace25 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:42.146179 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 595/1206] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:45:42.152191 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d3e4a-13e0-45aa-bd83-424a7c4d06c0 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:42.156777 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d3e4a-13e0-45aa-bd83-424a7c4d06c0 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:42.156777 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d3e4a-13e0-45aa-bd83-424a7c4d06c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:42.171588 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-977d3e4a-13e0-45aa-bd83-424a7c4d06c0 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:45:42.172777 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-977d3e4a-13e0-45aa-bd83-424a7c4d06c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:45:42.238428 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 612/1207] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:45:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:45:42.516446 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ddef38-f739-4974-949b-3bb24d0ace25 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:42.617936 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ddef38-f739-4974-949b-3bb24d0ace25 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:42.657829 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a589194-bf5c-47f2-9536-b7d8bd199f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:42.658446 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a589194-bf5c-47f2-9536-b7d8bd199f07 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:42.658751 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a589194-bf5c-47f2-9536-b7d8bd199f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:42.659021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a589194-bf5c-47f2-9536-b7d8bd199f07 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:42.669267 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a589194-bf5c-47f2-9536-b7d8bd199f07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:45:42.674014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-6a589194-bf5c-47f2-9536-b7d8bd199f07 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:45:42.674014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:45:42.674014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:45:42.674014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:45:42.674014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:45:42.674014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:45:42.674014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:45:42.674014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:45:42.674014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:45:42.674014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:45:42.674014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:45:42.674014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:45:42.674014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:45:42.674014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:45:42.674014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:45:42.674014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:45:42.675160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:45:42.675160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:45:42.675160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:45:42.675160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:45:42.675160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:45:42.675160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:45:42.675160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:45:42.675160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:45:42.675160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:45:42.675160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:45:42.675160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:45:42.675160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:45:42.675160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:45:42.675160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:45:42.675160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:45:42.675160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:45:42.676127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:45:42.676127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:45:42.676127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:45:42.676127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:45:42.676127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:45:42.676127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:45:42.676127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:45:42.676127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:45:42.676127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:45:42.676127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:45:42.676127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:45:42.676127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:45:42.676127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 596/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Sep 10 23:45:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:45:42.681155 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977d3e4a-13e0-45aa-bd83-424a7c4d06c0 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:42.720915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:42.779968 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977d3e4a-13e0-45aa-bd83-424a7c4d06c0 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:42.814063 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597d5be-3bb2-4135-bbeb-ad8619dedf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:42.814674 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597d5be-3bb2-4135-bbeb-ad8619dedf0d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:42.814962 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597d5be-3bb2-4135-bbeb-ad8619dedf0d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:42.815231 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597d5be-3bb2-4135-bbeb-ad8619dedf0d None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:42.815880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597d5be-3bb2-4135-bbeb-ad8619dedf0d None admin] RBAC: Adding query filter params (name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:45:42.816178 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597d5be-3bb2-4135-bbeb-ad8619dedf0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:42.817525 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597d5be-3bb2-4135-bbeb-ad8619dedf0d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:42.825609 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 613/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Sep 10 23:45:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:45:42.831458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a589194-bf5c-47f2-9536-b7d8bd199f07 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:42.881459 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:42.929223 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a589194-bf5c-47f2-9536-b7d8bd199f07 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:42.962979 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389273cd-9651-468d-8388-cfa328771bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:42.963602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389273cd-9651-468d-8388-cfa328771bee None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:42.963883 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389273cd-9651-468d-8388-cfa328771bee None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:42.964158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389273cd-9651-468d-8388-cfa328771bee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:42.974892 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389273cd-9651-468d-8388-cfa328771bee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:42.976343 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389273cd-9651-468d-8388-cfa328771bee None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:42.990969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-389273cd-9651-468d-8388-cfa328771bee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:45:42.990969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:45:42.990969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:45:42.990969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:45:42.990969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:45:42.990969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:45:42.990969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:45:42.990969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:45:42.990969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:45:42.990969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:45:42.990969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:45:42.990969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:45:42.990969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:45:42.990969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:45:42.990969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:45:42.990969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:45:43.003996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:45:43.003996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:45:43.003996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:45:43.003996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:45:43.003996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:45:43.003996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:45:43.003996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:45:43.003996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:45:43.003996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:45:43.003996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:45:43.003996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:45:43.003996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:45:43.003996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:45:43.003996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:45:43.003996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:45:43.003996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:45:43.003996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:45:43.005086 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:45:43.005086 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:45:43.005086 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:45:43.005086 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 597/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:45:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:45:43.005317 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597d5be-3bb2-4135-bbeb-ad8619dedf0d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:43.093712 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:43.096572 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597d5be-3bb2-4135-bbeb-ad8619dedf0d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:43.129657 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775441b6-5ae8-4a5a-8607-485ae012a193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:43.130243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775441b6-5ae8-4a5a-8607-485ae012a193 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:43.130516 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775441b6-5ae8-4a5a-8607-485ae012a193 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:43.130816 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775441b6-5ae8-4a5a-8607-485ae012a193 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:43.131443 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775441b6-5ae8-4a5a-8607-485ae012a193 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:45:43.131736 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775441b6-5ae8-4a5a-8607-485ae012a193 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:43.133068 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775441b6-5ae8-4a5a-8607-485ae012a193 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:43.144810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 614/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:45:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:45:43.150601 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389273cd-9651-468d-8388-cfa328771bee None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:43.249318 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389273cd-9651-468d-8388-cfa328771bee None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:43.250851 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:43.282416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266600f-627c-4ed6-aa2e-51396ee4340a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:43.283056 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266600f-627c-4ed6-aa2e-51396ee4340a None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:43.283336 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266600f-627c-4ed6-aa2e-51396ee4340a None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:43.283609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266600f-627c-4ed6-aa2e-51396ee4340a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:43.290307 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266600f-627c-4ed6-aa2e-51396ee4340a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:43.292326 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266600f-627c-4ed6-aa2e-51396ee4340a None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:43.303706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-b266600f-627c-4ed6-aa2e-51396ee4340a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 10 23:45:43.303706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:45:43.303706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:45:43.303706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:45:43.303706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:45:43.303706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:45:43.303706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:45:43.303706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:45:43.303706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:45:43.303706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:45:43.303706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:45:43.303706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:45:43.303706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:45:43.303706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:45:43.303706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:45:43.303706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:45:43.304743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:45:43.304743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:45:43.304743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:45:43.304743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:45:43.304743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:45:43.304743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:45:43.304743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:45:43.304743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:45:43.304743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:45:43.304743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:45:43.304743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:45:43.304743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:45:43.304743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:45:43.304743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:45:43.304743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:45:43.304743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:45:43.304743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:45:43.304743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:45:43.305763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:45:43.305763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:45:43.305763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:45:43.305763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:45:43.305763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:45:43.305763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:45:43.305763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:45:43.305763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:45:43.305763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 10 23:45:43.305763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:45:43.305763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 598/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:45:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:45:43.311019 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775441b6-5ae8-4a5a-8607-485ae012a193 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:43.410247 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775441b6-5ae8-4a5a-8607-485ae012a193 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:43.443785 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d063d6e-92cc-4865-b78d-977ad44dbcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:43.444380 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d063d6e-92cc-4865-b78d-977ad44dbcfe None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:43.444657 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d063d6e-92cc-4865-b78d-977ad44dbcfe None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:43.444929 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d063d6e-92cc-4865-b78d-977ad44dbcfe None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:43.445645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d063d6e-92cc-4865-b78d-977ad44dbcfe None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:45:43.445948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d063d6e-92cc-4865-b78d-977ad44dbcfe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:43.447463 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d063d6e-92cc-4865-b78d-977ad44dbcfe None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:43.464888 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 615/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Sat Sep 10 23:45:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:45:43.470921 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266600f-627c-4ed6-aa2e-51396ee4340a None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:43.569826 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266600f-627c-4ed6-aa2e-51396ee4340a None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:43.604014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750c713-e925-4c3b-8763-d03758e2b044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:43.604985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750c713-e925-4c3b-8763-d03758e2b044 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:43.605468 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750c713-e925-4c3b-8763-d03758e2b044 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:43.605959 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750c713-e925-4c3b-8763-d03758e2b044 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:43.641117 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:43.678880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6750c713-e925-4c3b-8763-d03758e2b044 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:43.680271 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6750c713-e925-4c3b-8763-d03758e2b044 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:43.781510 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 599/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:45:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 23:45:43.787440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d063d6e-92cc-4865-b78d-977ad44dbcfe None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:43.805658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:43.883958 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d063d6e-92cc-4865-b78d-977ad44dbcfe None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:43.917983 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cd28c-8219-4c0f-9541-ffbf1f67c03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:43.918594 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cd28c-8219-4c0f-9541-ffbf1f67c03d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:43.918894 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cd28c-8219-4c0f-9541-ffbf1f67c03d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:43.919160 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cd28c-8219-4c0f-9541-ffbf1f67c03d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:43.928937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28cd28c-8219-4c0f-9541-ffbf1f67c03d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:43.930289 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28cd28c-8219-4c0f-9541-ffbf1f67c03d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:43.946083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-e28cd28c-8219-4c0f-9541-ffbf1f67c03d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:45:43.946083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:45:43.946083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:45:43.946083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:45:43.946083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:45:43.946083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:45:43.946083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:45:43.946083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:45:43.946083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:45:43.946083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:45:43.946083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:45:43.946083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:45:43.946083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:45:43.946083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:45:43.946083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:45:43.946083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:45:43.947253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:45:43.947253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:45:43.947253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:45:43.947253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:45:43.947253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:45:43.947253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:45:43.947253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:45:43.947253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:45:43.947253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:45:43.947253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:45:43.947253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:45:43.947253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:45:43.947253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:45:43.947253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:45:43.947253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:45:43.947253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:45:43.947253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:45:43.948874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:45:43.948874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:45:43.948874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:45:43.948874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 616/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:45:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:45:43.953333 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750c713-e925-4c3b-8763-d03758e2b044 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:44.015330 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:44.052916 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750c713-e925-4c3b-8763-d03758e2b044 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:44.089012 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d257c-be68-4539-a66b-b83f1bbe238e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:44.089640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d257c-be68-4539-a66b-b83f1bbe238e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:44.089899 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d257c-be68-4539-a66b-b83f1bbe238e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:44.090166 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d257c-be68-4539-a66b-b83f1bbe238e None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:44.090826 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d257c-be68-4539-a66b-b83f1bbe238e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:45:44.091127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d257c-be68-4539-a66b-b83f1bbe238e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:44.092480 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d257c-be68-4539-a66b-b83f1bbe238e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:44.103959 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 600/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:45:44.110005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cd28c-8219-4c0f-9541-ffbf1f67c03d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:44.178230 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:44.208931 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cd28c-8219-4c0f-9541-ffbf1f67c03d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:44.243760 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b381fb-8f6c-405e-95de-7d6bec1f6123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:44.244399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b381fb-8f6c-405e-95de-7d6bec1f6123 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:44.244672 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b381fb-8f6c-405e-95de-7d6bec1f6123 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:44.244933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b381fb-8f6c-405e-95de-7d6bec1f6123 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:44.251539 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b381fb-8f6c-405e-95de-7d6bec1f6123 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:44.252958 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b381fb-8f6c-405e-95de-7d6bec1f6123 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:44.264288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-65b381fb-8f6c-405e-95de-7d6bec1f6123 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:45:44.264288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:45:44.264288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:45:44.264288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:45:44.264288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:45:44.264288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:45:44.264288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:45:44.264288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:45:44.264288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:45:44.264288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:45:44.264288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:45:44.264288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:45:44.264288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:45:44.264288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:45:44.264288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:45:44.265330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:45:44.265330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:45:44.265330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:45:44.265330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:45:44.265330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:45:44.265330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:45:44.265330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:45:44.265330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:45:44.265330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:45:44.265330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:45:44.265330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:45:44.265330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:45:44.265330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:45:44.265330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:45:44.265330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:45:44.265330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:45:44.265330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:45:44.266409 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:45:44.266409 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:45:44.266409 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:45:44.266409 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:45:44.266409 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:45:44.266409 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:45:44.266409 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:45:44.266409 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:45:44.266409 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 617/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:45:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:45:44.271520 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d257c-be68-4539-a66b-b83f1bbe238e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:44.370343 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d257c-be68-4539-a66b-b83f1bbe238e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:44.393670 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:44.404967 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7167550-14a4-4838-9ca5-141d83242353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:44.405556 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7167550-14a4-4838-9ca5-141d83242353 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:44.405830 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7167550-14a4-4838-9ca5-141d83242353 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:44.406097 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7167550-14a4-4838-9ca5-141d83242353 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:44.406799 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7167550-14a4-4838-9ca5-141d83242353 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:45:44.407098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7167550-14a4-4838-9ca5-141d83242353 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:44.408518 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7167550-14a4-4838-9ca5-141d83242353 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:44.419476 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 601/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:45:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:45:44.425704 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b381fb-8f6c-405e-95de-7d6bec1f6123 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:44.528836 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b381fb-8f6c-405e-95de-7d6bec1f6123 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:44.555950 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:44.561824 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b33ea-1948-4240-b72f-72915fd5b211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:44.562411 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b33ea-1948-4240-b72f-72915fd5b211 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:44.562705 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b33ea-1948-4240-b72f-72915fd5b211 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:44.562976 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b33ea-1948-4240-b72f-72915fd5b211 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:44.563685 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b33ea-1948-4240-b72f-72915fd5b211 None admin] RBAC: Adding query filter params (role.id=8ff1727b65104e5fb97333d477196d46, scope.project.id=947277c4458645899287e6818947c076, user.id=5cb1d85a338e40f18e61887a1becb906) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:45:44.563985 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b33ea-1948-4240-b72f-72915fd5b211 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:44.565397 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b33ea-1948-4240-b72f-72915fd5b211 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:44.570440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 618/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:45:44 2022] GET /identity/v3/role_assignments?user.id=5cb1d85a338e40f18e61887a1becb906&scope.project.id=947277c4458645899287e6818947c076&role.id=8ff1727b65104e5fb97333d477196d46 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:45:46.684623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6ed87-fb67-4295-8bbd-f9a4fbe483ee None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:46.685173 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6ed87-fb67-4295-8bbd-f9a4fbe483ee None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:46.685173 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6ed87-fb67-4295-8bbd-f9a4fbe483ee None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:46.686507 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 602/1220] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:45:46.691586 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed8394-4586-4de2-8459-3ba1d3b028c0 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:46.692011 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed8394-4586-4de2-8459-3ba1d3b028c0 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:46.692103 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed8394-4586-4de2-8459-3ba1d3b028c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:46.709819 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-1bed8394-4586-4de2-8459-3ba1d3b028c0 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:45:46.711018 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1bed8394-4586-4de2-8459-3ba1d3b028c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:45:46.775115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 619/1221] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:45:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:45:47.045743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6ed87-fb67-4295-8bbd-f9a4fbe483ee None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:47.148582 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6ed87-fb67-4295-8bbd-f9a4fbe483ee None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:47.182250 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e95ee78-8bf8-4759-acc4-5053ee274a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:47.183147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95ee78-8bf8-4759-acc4-5053ee274a50 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:47.183422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95ee78-8bf8-4759-acc4-5053ee274a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:47.183688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95ee78-8bf8-4759-acc4-5053ee274a50 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:47.193840 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e95ee78-8bf8-4759-acc4-5053ee274a50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:45:47.198894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-5e95ee78-8bf8-4759-acc4-5053ee274a50 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:45:47.198894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:45:47.198894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:45:47.198894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:45:47.198894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:45:47.198894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:45:47.198894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:45:47.198894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:45:47.198894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:45:47.198894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:45:47.198894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:45:47.198894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:45:47.198894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:45:47.198894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:45:47.198894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:45:47.198894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:45:47.199993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:45:47.199993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:45:47.199993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:45:47.199993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:45:47.199993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:45:47.199993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:45:47.199993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:45:47.199993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:45:47.199993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:45:47.199993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:45:47.199993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:45:47.199993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:45:47.199993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:45:47.199993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:45:47.199993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:45:47.199993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:45:47.200894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:45:47.200894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:45:47.200894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:45:47.200894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:45:47.200894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:45:47.200894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:45:47.200894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:45:47.200894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:45:47.200894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:45:47.200894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:45:47.200894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:45:47.200894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:45:47.200894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 603/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Sep 10 23:45:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:45:47.206121 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed8394-4586-4de2-8459-3ba1d3b028c0 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:47.310957 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed8394-4586-4de2-8459-3ba1d3b028c0 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:47.333864 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:47.345047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf92947-75be-4416-a36b-9fd3c9e31804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:47.345705 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf92947-75be-4416-a36b-9fd3c9e31804 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:47.345954 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf92947-75be-4416-a36b-9fd3c9e31804 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:47.346217 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf92947-75be-4416-a36b-9fd3c9e31804 None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:47.346898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf92947-75be-4416-a36b-9fd3c9e31804 None admin] RBAC: Adding query filter params (name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:45:47.347192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf92947-75be-4416-a36b-9fd3c9e31804 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:47.348603 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf92947-75be-4416-a36b-9fd3c9e31804 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:47.356259 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 620/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Sep 10 23:45:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:45:47.362108 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e95ee78-8bf8-4759-acc4-5053ee274a50 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:47.409689 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:47.463689 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e95ee78-8bf8-4759-acc4-5053ee274a50 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:47.498251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d930b-16fe-4e20-a32c-f8d64c135d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:47.498924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d930b-16fe-4e20-a32c-f8d64c135d14 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:47.499195 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d930b-16fe-4e20-a32c-f8d64c135d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:47.499459 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d930b-16fe-4e20-a32c-f8d64c135d14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:47.509119 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d930b-16fe-4e20-a32c-f8d64c135d14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:47.511299 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d930b-16fe-4e20-a32c-f8d64c135d14 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:47.526006 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-d23d930b-16fe-4e20-a32c-f8d64c135d14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:45:47.526006 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:45:47.526006 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:45:47.526006 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:45:47.526006 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:45:47.526006 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:45:47.526006 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:45:47.526006 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:45:47.526006 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:45:47.526006 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:45:47.526006 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:45:47.526006 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:45:47.526006 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:45:47.526006 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:45:47.526006 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:45:47.526006 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:45:47.527164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:45:47.527164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:45:47.527164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:45:47.527164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:45:47.527164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:45:47.527164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:45:47.527164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:45:47.527164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:45:47.527164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:45:47.527164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:45:47.527164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:45:47.527164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:45:47.527164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:45:47.527164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:45:47.527164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:45:47.527164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:45:47.527164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:45:47.528252 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:45:47.528252 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:45:47.528252 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:45:47.528252 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 604/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:45:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:45:47.533135 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf92947-75be-4416-a36b-9fd3c9e31804 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:47.632160 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf92947-75be-4416-a36b-9fd3c9e31804 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:47.665565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6b7d9-04b7-41b4-8076-9953135ee1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:47.666488 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6b7d9-04b7-41b4-8076-9953135ee1ac None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:47.666805 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6b7d9-04b7-41b4-8076-9953135ee1ac None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:47.667073 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6b7d9-04b7-41b4-8076-9953135ee1ac None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:47.667701 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d6b7d9-04b7-41b4-8076-9953135ee1ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:45:47.667997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d6b7d9-04b7-41b4-8076-9953135ee1ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:47.669336 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d6b7d9-04b7-41b4-8076-9953135ee1ac None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:47.681271 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 621/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:45:47.687920 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d930b-16fe-4e20-a32c-f8d64c135d14 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:47.780547 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:47.786996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d930b-16fe-4e20-a32c-f8d64c135d14 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:47.820552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa33df0-8d9e-495a-91f9-dfb87a10e950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:47.821163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa33df0-8d9e-495a-91f9-dfb87a10e950 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:47.821439 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa33df0-8d9e-495a-91f9-dfb87a10e950 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:47.821706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa33df0-8d9e-495a-91f9-dfb87a10e950 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:47.829480 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa33df0-8d9e-495a-91f9-dfb87a10e950 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:47.831065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa33df0-8d9e-495a-91f9-dfb87a10e950 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:47.842405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-efa33df0-8d9e-495a-91f9-dfb87a10e950 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 10 23:45:47.842405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:45:47.842405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:45:47.842405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:45:47.842405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:45:47.842405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:45:47.842405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:45:47.842405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:45:47.842405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:45:47.842405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:45:47.842405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:45:47.842405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:45:47.842405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:45:47.842405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:45:47.842405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:45:47.842405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:45:47.843474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:45:47.843474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:45:47.843474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:45:47.843474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:45:47.843474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:45:47.843474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:45:47.843474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:45:47.843474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:45:47.843474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:45:47.843474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:45:47.843474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:45:47.843474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:45:47.843474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:45:47.843474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:45:47.843474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:45:47.843474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:45:47.843474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:45:47.843474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:45:47.844401 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:45:47.844401 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:45:47.844401 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:45:47.844401 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:45:47.844401 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:45:47.844401 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:45:47.844401 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:45:47.844401 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:45:47.844401 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 10 23:45:47.844401 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:45:47.844401 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 605/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:45:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:45:47.849539 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6b7d9-04b7-41b4-8076-9953135ee1ac None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:47.896148 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:47.948710 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6b7d9-04b7-41b4-8076-9953135ee1ac None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:47.983764 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f1bdd-0602-4a41-9ec5-8c3093d12232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:47.984375 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f1bdd-0602-4a41-9ec5-8c3093d12232 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:47.984647 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f1bdd-0602-4a41-9ec5-8c3093d12232 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:47.984912 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f1bdd-0602-4a41-9ec5-8c3093d12232 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:47.985626 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f1bdd-0602-4a41-9ec5-8c3093d12232 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:45:47.985922 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f1bdd-0602-4a41-9ec5-8c3093d12232 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:47.987347 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f1bdd-0602-4a41-9ec5-8c3093d12232 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:48.004377 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 622/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Sat Sep 10 23:45:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:45:48.010248 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa33df0-8d9e-495a-91f9-dfb87a10e950 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:48.108740 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa33df0-8d9e-495a-91f9-dfb87a10e950 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:48.141970 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30614af1-d10a-4524-8a2a-4dfdaed50a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:48.142572 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30614af1-d10a-4524-8a2a-4dfdaed50a85 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:48.142872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30614af1-d10a-4524-8a2a-4dfdaed50a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:48.143136 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30614af1-d10a-4524-8a2a-4dfdaed50a85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:48.154284 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30614af1-d10a-4524-8a2a-4dfdaed50a85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:48.155686 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30614af1-d10a-4524-8a2a-4dfdaed50a85 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:48.170152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-30614af1-d10a-4524-8a2a-4dfdaed50a85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:45:48.170152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:45:48.170152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:45:48.170152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:45:48.170152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:45:48.170152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:45:48.170152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:45:48.170152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:45:48.170152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:45:48.170152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:45:48.170152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:45:48.170152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:45:48.170152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:45:48.170152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:45:48.170152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:45:48.170152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:45:48.171331 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:45:48.171331 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:45:48.171331 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:45:48.171331 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:45:48.171331 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:45:48.171331 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:45:48.171331 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:45:48.171331 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:45:48.171331 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:45:48.171331 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:45:48.171331 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:45:48.171331 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:45:48.171331 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:45:48.171331 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:45:48.171331 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:45:48.171331 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:45:48.171331 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:45:48.172456 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:45:48.172456 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:45:48.172456 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:45:48.172456 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 606/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:45:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:45:48.177244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f1bdd-0602-4a41-9ec5-8c3093d12232 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:48.277791 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f1bdd-0602-4a41-9ec5-8c3093d12232 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:48.280109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:48.313077 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d30a10-3478-4f82-bf2f-decd4d0f59ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:48.313672 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d30a10-3478-4f82-bf2f-decd4d0f59ac None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:48.313946 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d30a10-3478-4f82-bf2f-decd4d0f59ac None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:48.314213 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d30a10-3478-4f82-bf2f-decd4d0f59ac None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:48.314919 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d30a10-3478-4f82-bf2f-decd4d0f59ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:45:48.315192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d30a10-3478-4f82-bf2f-decd4d0f59ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:48.316536 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d30a10-3478-4f82-bf2f-decd4d0f59ac None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:48.328560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 623/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:45:48.334386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30614af1-d10a-4524-8a2a-4dfdaed50a85 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:48.344255 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:48.435482 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30614af1-d10a-4524-8a2a-4dfdaed50a85 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:48.468469 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da26c4-49af-4fab-9249-b13ea48070b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:48.469082 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da26c4-49af-4fab-9249-b13ea48070b5 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:48.469356 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da26c4-49af-4fab-9249-b13ea48070b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:48.469624 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da26c4-49af-4fab-9249-b13ea48070b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:48.476182 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da26c4-49af-4fab-9249-b13ea48070b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:48.477623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da26c4-49af-4fab-9249-b13ea48070b5 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:48.488900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-90da26c4-49af-4fab-9249-b13ea48070b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:45:48.488900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:45:48.488900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:45:48.488900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:45:48.488900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:45:48.488900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:45:48.488900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:45:48.488900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:45:48.488900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:45:48.488900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:45:48.488900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:45:48.488900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:45:48.488900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:45:48.488900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:45:48.488900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:45:48.489812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:45:48.489812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:45:48.489812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:45:48.489812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:45:48.489812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:45:48.489812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:45:48.489812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:45:48.489812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:45:48.489812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:45:48.489812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:45:48.489812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:45:48.489812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:45:48.489812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:45:48.489812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:45:48.489812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:45:48.489812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:45:48.489812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:45:48.490808 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:45:48.490808 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:45:48.490808 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:45:48.490808 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:45:48.490808 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:45:48.490808 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:45:48.490808 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:45:48.490808 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:45:48.490808 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 607/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:45:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:45:48.496854 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d30a10-3478-4f82-bf2f-decd4d0f59ac None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:48.595155 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d30a10-3478-4f82-bf2f-decd4d0f59ac None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:48.628683 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c067e20-7016-46b7-8ddb-36832a7dfd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:48.629313 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c067e20-7016-46b7-8ddb-36832a7dfd1f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:48.629563 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c067e20-7016-46b7-8ddb-36832a7dfd1f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:48.629827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c067e20-7016-46b7-8ddb-36832a7dfd1f None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:48.630512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c067e20-7016-46b7-8ddb-36832a7dfd1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:45:48.630842 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c067e20-7016-46b7-8ddb-36832a7dfd1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:48.632261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c067e20-7016-46b7-8ddb-36832a7dfd1f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:48.645162 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 624/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:45:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:45:48.651571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da26c4-49af-4fab-9249-b13ea48070b5 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:48.720620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:48.754538 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da26c4-49af-4fab-9249-b13ea48070b5 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:48.788167 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6983b-eb24-42e9-9226-ef939d7e0b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:48.788807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6983b-eb24-42e9-9226-ef939d7e0b83 None admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:48.789084 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6983b-eb24-42e9-9226-ef939d7e0b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:48.789356 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6983b-eb24-42e9-9226-ef939d7e0b83 None admin] PATH_INFO: `/v3/projects/947277c4458645899287e6818947c076/users/5cb1d85a338e40f18e61887a1becb906/roles/8ff1727b65104e5fb97333d477196d46` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:48.803554 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6983b-eb24-42e9-9226-ef939d7e0b83 None admin] RBAC: Authorizing `identity:create_grant(project_id=947277c4458645899287e6818947c076, user_id=5cb1d85a338e40f18e61887a1becb906, role_id=8ff1727b65104e5fb97333d477196d46)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:48.808687 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6983b-eb24-42e9-9226-ef939d7e0b83 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:48.883624 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 608/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Sep 10 23:45:48 2022] PUT /identity/v3/projects/947277c4458645899287e6818947c076/users/5cb1d85a338e40f18e61887a1becb906/roles/8ff1727b65104e5fb97333d477196d46 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:45:50.996381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47af351-3470-4c31-a947-25b5a3f25b08 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:50.997280 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47af351-3470-4c31-a947-25b5a3f25b08 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:50.997280 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47af351-3470-4c31-a947-25b5a3f25b08 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:50.998757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 625/1233] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:45:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:45:51.003865 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bf8fe-192a-4d61-94b3-561684ef17ee None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:51.004332 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bf8fe-192a-4d61-94b3-561684ef17ee None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:51.004555 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bf8fe-192a-4d61-94b3-561684ef17ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:51.022558 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c68bf8fe-192a-4d61-94b3-561684ef17ee None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:45:51.023775 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c68bf8fe-192a-4d61-94b3-561684ef17ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:45:51.090909 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 609/1234] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:45:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:45:51.357803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47af351-3470-4c31-a947-25b5a3f25b08 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:51.382223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:51.458683 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47af351-3470-4c31-a947-25b5a3f25b08 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:51.490302 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44316e9e-a2c2-42da-ba06-ed79f45e0e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:51.490935 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44316e9e-a2c2-42da-ba06-ed79f45e0e30 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:51.491214 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44316e9e-a2c2-42da-ba06-ed79f45e0e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:51.491467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44316e9e-a2c2-42da-ba06-ed79f45e0e30 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:51.501652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44316e9e-a2c2-42da-ba06-ed79f45e0e30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:45:51.506368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-44316e9e-a2c2-42da-ba06-ed79f45e0e30 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:45:51.506368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:45:51.506368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:45:51.506368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:45:51.506368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:45:51.506368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:45:51.506368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:45:51.506368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:45:51.506368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:45:51.506368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:45:51.506368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:45:51.506368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:45:51.506368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:45:51.506368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:45:51.506368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:45:51.506368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:45:51.511123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:45:51.511123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:45:51.511123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:45:51.511123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:45:51.511123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:45:51.511123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:45:51.511123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:45:51.511123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:45:51.511123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:45:51.511123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:45:51.511123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:45:51.511123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:45:51.511123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:45:51.511123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:45:51.511123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:45:51.511123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:45:51.512166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:45:51.512166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:45:51.512166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:45:51.512166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:45:51.512166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:45:51.512166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:45:51.512166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:45:51.512166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:45:51.512166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:45:51.512166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:45:51.512166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 23:45:51.512166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:45:51.512166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 626/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Sat Sep 10 23:45:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:45:51.514742 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bf8fe-192a-4d61-94b3-561684ef17ee None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:51.560666 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:51.610796 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bf8fe-192a-4d61-94b3-561684ef17ee None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:51.658087 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d6707-22fb-4032-9e07-222b7d243b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:51.658691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d6707-22fb-4032-9e07-222b7d243b3e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:51.658973 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d6707-22fb-4032-9e07-222b7d243b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:51.659238 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d6707-22fb-4032-9e07-222b7d243b3e None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:51.659913 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d6707-22fb-4032-9e07-222b7d243b3e None admin] RBAC: Adding query filter params (name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:45:51.660204 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d6707-22fb-4032-9e07-222b7d243b3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:51.661556 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d6707-22fb-4032-9e07-222b7d243b3e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:51.669374 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 610/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Sat Sep 10 23:45:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:45:51.675385 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44316e9e-a2c2-42da-ba06-ed79f45e0e30 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:51.761361 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:51.775946 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44316e9e-a2c2-42da-ba06-ed79f45e0e30 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:51.809186 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d1e215-73cc-4dc1-ac5c-048c0873ebaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:51.809831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1e215-73cc-4dc1-ac5c-048c0873ebaa None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:51.810095 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1e215-73cc-4dc1-ac5c-048c0873ebaa None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:51.810361 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1e215-73cc-4dc1-ac5c-048c0873ebaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:51.820937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d1e215-73cc-4dc1-ac5c-048c0873ebaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:51.821428 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d1e215-73cc-4dc1-ac5c-048c0873ebaa None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:51.836190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-88d1e215-73cc-4dc1-ac5c-048c0873ebaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:45:51.836190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:45:51.836190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:45:51.836190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:45:51.836190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:45:51.836190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:45:51.836190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:45:51.836190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:45:51.836190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:45:51.836190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:45:51.836190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:45:51.836190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:45:51.836190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:45:51.836190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:45:51.836190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:45:51.836190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:45:51.837314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:45:51.837314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:45:51.837314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:45:51.837314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:45:51.837314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:45:51.837314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:45:51.837314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:45:51.837314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:45:51.837314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:45:51.837314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:45:51.837314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:45:51.837314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:45:51.837314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:45:51.837314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:45:51.837314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:45:51.837314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:45:51.837314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:45:51.838383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:45:51.838383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:45:51.838383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:45:51.839008 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 627/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:45:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:45:51.843275 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d6707-22fb-4032-9e07-222b7d243b3e None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:51.943743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d6707-22fb-4032-9e07-222b7d243b3e None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:51.952666 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:51.977034 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150541fc-96c0-4807-8efa-c04e75ca02d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:51.977623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150541fc-96c0-4807-8efa-c04e75ca02d4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:51.977900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150541fc-96c0-4807-8efa-c04e75ca02d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:51.978734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150541fc-96c0-4807-8efa-c04e75ca02d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:51.979371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150541fc-96c0-4807-8efa-c04e75ca02d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:45:51.979670 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150541fc-96c0-4807-8efa-c04e75ca02d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:51.981038 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150541fc-96c0-4807-8efa-c04e75ca02d4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:51.992344 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 611/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:45:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:45:51.998411 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d1e215-73cc-4dc1-ac5c-048c0873ebaa None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:52.098561 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d1e215-73cc-4dc1-ac5c-048c0873ebaa None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:52.131169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc58ff4-ad72-435a-8a3b-7b2da1744f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:52.131779 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc58ff4-ad72-435a-8a3b-7b2da1744f90 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:52.132053 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc58ff4-ad72-435a-8a3b-7b2da1744f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:52.132322 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc58ff4-ad72-435a-8a3b-7b2da1744f90 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:52.137061 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:52.141854 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc58ff4-ad72-435a-8a3b-7b2da1744f90 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:52.143303 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc58ff4-ad72-435a-8a3b-7b2da1744f90 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:52.155404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-7bc58ff4-ad72-435a-8a3b-7b2da1744f90 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 10 23:45:52.155404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:45:52.155404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:45:52.155404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:45:52.155404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:45:52.155404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:45:52.155404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:45:52.155404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:45:52.155404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:45:52.155404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:45:52.155404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:45:52.155404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:45:52.155404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:45:52.155404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:45:52.155404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:45:52.155404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:45:52.156521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:45:52.156521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:45:52.156521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:45:52.156521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:45:52.156521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:45:52.156521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:45:52.156521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:45:52.156521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:45:52.156521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:45:52.156521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:45:52.156521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:45:52.156521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:45:52.156521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:45:52.156521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:45:52.156521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:45:52.156521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:45:52.156521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:45:52.156521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:45:52.157925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:45:52.157925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:45:52.157925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:45:52.157925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:45:52.157925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:45:52.157925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:45:52.157925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:45:52.157925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:45:52.157925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 10 23:45:52.157925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:45:52.157925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 628/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:45:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:45:52.162293 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150541fc-96c0-4807-8efa-c04e75ca02d4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:52.262059 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150541fc-96c0-4807-8efa-c04e75ca02d4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:52.294991 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e0f45-6647-4ea5-ac2c-7091cb39f34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:52.295583 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e0f45-6647-4ea5-ac2c-7091cb39f34c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:52.295855 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e0f45-6647-4ea5-ac2c-7091cb39f34c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:52.296121 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e0f45-6647-4ea5-ac2c-7091cb39f34c None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:52.296831 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e0f45-6647-4ea5-ac2c-7091cb39f34c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:45:52.297129 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e0f45-6647-4ea5-ac2c-7091cb39f34c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:52.298538 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e0f45-6647-4ea5-ac2c-7091cb39f34c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:52.317037 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 612/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Sat Sep 10 23:45:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:45:52.323723 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc58ff4-ad72-435a-8a3b-7b2da1744f90 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:52.420722 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc58ff4-ad72-435a-8a3b-7b2da1744f90 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:52.453197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdde17-cd9f-494e-82cd-af7819467dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:52.453791 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdde17-cd9f-494e-82cd-af7819467dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:52.454066 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdde17-cd9f-494e-82cd-af7819467dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:52.454335 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdde17-cd9f-494e-82cd-af7819467dd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:52.513529 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:52.519939 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bdde17-cd9f-494e-82cd-af7819467dd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:52.521382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bdde17-cd9f-494e-82cd-af7819467dd0 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:52.617437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 629/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:45:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 23:45:52.623418 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e0f45-6647-4ea5-ac2c-7091cb39f34c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:52.653758 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:52.724030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e0f45-6647-4ea5-ac2c-7091cb39f34c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:52.757844 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540a684-a6a4-4ac6-8e04-35358c2752dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:52.758459 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540a684-a6a4-4ac6-8e04-35358c2752dd None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:52.758767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540a684-a6a4-4ac6-8e04-35358c2752dd None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:52.759036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540a684-a6a4-4ac6-8e04-35358c2752dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:52.769314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0540a684-a6a4-4ac6-8e04-35358c2752dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:52.770780 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0540a684-a6a4-4ac6-8e04-35358c2752dd None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:52.785861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-0540a684-a6a4-4ac6-8e04-35358c2752dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:45:52.785861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:45:52.785861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:45:52.785861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:45:52.785861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:45:52.785861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:45:52.785861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:45:52.785861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:45:52.785861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:45:52.785861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:45:52.785861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:45:52.785861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:45:52.785861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:45:52.785861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:45:52.785861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:45:52.785861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:45:52.786858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:45:52.786858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:45:52.786858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:45:52.786858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:45:52.786858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:45:52.786858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:45:52.786858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:45:52.786858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:45:52.786858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:45:52.786858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:45:52.786858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:45:52.786858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:45:52.786858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:45:52.786858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:45:52.786858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:45:52.786858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:45:52.786858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:45:52.787774 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:45:52.787774 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:45:52.787774 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:45:52.787774 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 613/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:45:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:45:52.792790 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdde17-cd9f-494e-82cd-af7819467dd0 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:52.885009 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:52.893371 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdde17-cd9f-494e-82cd-af7819467dd0 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:52.927348 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a0480-637b-4388-b48c-c2fc06d5b869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:52.928040 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a0480-637b-4388-b48c-c2fc06d5b869 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:52.928316 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a0480-637b-4388-b48c-c2fc06d5b869 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:52.928582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a0480-637b-4388-b48c-c2fc06d5b869 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:52.929210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a0480-637b-4388-b48c-c2fc06d5b869 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:45:52.929513 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a0480-637b-4388-b48c-c2fc06d5b869 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:52.930892 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a0480-637b-4388-b48c-c2fc06d5b869 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:52.942608 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 630/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:45:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:45:52.948856 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540a684-a6a4-4ac6-8e04-35358c2752dd None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:53.026929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:53.047668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540a684-a6a4-4ac6-8e04-35358c2752dd None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:53.080312 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16211d7-d004-4bcf-bdd4-58342aa601e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:53.080916 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16211d7-d004-4bcf-bdd4-58342aa601e6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:53.081188 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16211d7-d004-4bcf-bdd4-58342aa601e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:53.081450 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16211d7-d004-4bcf-bdd4-58342aa601e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:53.088906 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16211d7-d004-4bcf-bdd4-58342aa601e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:53.090342 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16211d7-d004-4bcf-bdd4-58342aa601e6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:53.101329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-e16211d7-d004-4bcf-bdd4-58342aa601e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:45:53.101329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:45:53.101329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:45:53.101329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:45:53.101329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:45:53.101329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:45:53.101329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:45:53.101329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:45:53.101329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:45:53.101329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:45:53.101329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:45:53.101329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:45:53.101329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:45:53.101329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:45:53.101329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:45:53.102377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:45:53.102377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:45:53.102377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:45:53.102377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:45:53.102377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:45:53.102377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:45:53.102377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:45:53.102377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:45:53.102377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:45:53.102377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:45:53.102377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:45:53.102377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:45:53.102377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:45:53.102377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:45:53.102377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:45:53.102377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:45:53.102377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:45:53.103433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:45:53.103433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:45:53.103433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:45:53.103433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:45:53.103433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:45:53.103433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:45:53.103433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:45:53.103433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:45:53.103433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 614/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:45:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:45:53.108316 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a0480-637b-4388-b48c-c2fc06d5b869 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:53.209760 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a0480-637b-4388-b48c-c2fc06d5b869 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:53.242734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a04f60-e5b3-47d1-9a60-c4bf2b1f18cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:53.243325 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a04f60-e5b3-47d1-9a60-c4bf2b1f18cd None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:53.243596 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a04f60-e5b3-47d1-9a60-c4bf2b1f18cd None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:53.244399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a04f60-e5b3-47d1-9a60-c4bf2b1f18cd None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:53.245066 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a04f60-e5b3-47d1-9a60-c4bf2b1f18cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:45:53.245362 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a04f60-e5b3-47d1-9a60-c4bf2b1f18cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:53.246791 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a04f60-e5b3-47d1-9a60-c4bf2b1f18cd None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:53.258225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 631/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:45:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:45:53.264878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16211d7-d004-4bcf-bdd4-58342aa601e6 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:53.363370 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16211d7-d004-4bcf-bdd4-58342aa601e6 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:53.395471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011ec2e-7542-4014-a514-66dc11cde48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:53.396051 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011ec2e-7542-4014-a514-66dc11cde48e None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:53.396324 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011ec2e-7542-4014-a514-66dc11cde48e None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:53.396595 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011ec2e-7542-4014-a514-66dc11cde48e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:53.397301 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011ec2e-7542-4014-a514-66dc11cde48e None admin] RBAC: Adding query filter params (role.id=8ff1727b65104e5fb97333d477196d46, scope.project.id=947277c4458645899287e6818947c076, user.id=5cb1d85a338e40f18e61887a1becb906) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:45:53.397598 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011ec2e-7542-4014-a514-66dc11cde48e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:53.399031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011ec2e-7542-4014-a514-66dc11cde48e None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:53.404584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 615/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:45:53 2022] GET /identity/v3/role_assignments?user.id=5cb1d85a338e40f18e61887a1becb906&scope.project.id=947277c4458645899287e6818947c076&role.id=8ff1727b65104e5fb97333d477196d46 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:45:55.625873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4193ee-28f0-4885-9b3a-8c29381bb5c3 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:55.626372 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4193ee-28f0-4885-9b3a-8c29381bb5c3 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:55.626372 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4193ee-28f0-4885-9b3a-8c29381bb5c3 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:55.627692 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 632/1247] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:45:55.632783 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37cebc-b413-4cf3-8865-5a7fa2e61ecb None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:55.633169 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37cebc-b413-4cf3-8865-5a7fa2e61ecb None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:55.633317 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37cebc-b413-4cf3-8865-5a7fa2e61ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:55.639640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:55.651542 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-9d37cebc-b413-4cf3-8865-5a7fa2e61ecb None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:45:55.652702 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9d37cebc-b413-4cf3-8865-5a7fa2e61ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:45:55.717086 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 616/1248] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:45:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:45:55.986462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4193ee-28f0-4885-9b3a-8c29381bb5c3 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:56.004689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:56.086961 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4193ee-28f0-4885-9b3a-8c29381bb5c3 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:56.121274 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e99c0a-238f-4397-a18b-3cbd67c6c8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:56.121888 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e99c0a-238f-4397-a18b-3cbd67c6c8c2 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:56.122162 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e99c0a-238f-4397-a18b-3cbd67c6c8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:56.122521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e99c0a-238f-4397-a18b-3cbd67c6c8c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:56.132340 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e99c0a-238f-4397-a18b-3cbd67c6c8c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:45:56.137216 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-b8e99c0a-238f-4397-a18b-3cbd67c6c8c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:45:56.137216 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:45:56.137216 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:45:56.137216 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:45:56.137216 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:45:56.137216 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:45:56.137216 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:45:56.137216 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:45:56.137216 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:45:56.137216 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:45:56.137216 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:45:56.137216 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:45:56.137216 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:45:56.137216 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:45:56.137216 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:45:56.137216 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:45:56.138369 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:45:56.138369 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:45:56.138369 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:45:56.138369 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:45:56.138369 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:45:56.138369 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:45:56.138369 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:45:56.138369 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:45:56.138369 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:45:56.138369 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:45:56.138369 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:45:56.138369 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:45:56.138369 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:45:56.138369 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:45:56.138369 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:45:56.138369 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:45:56.139428 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:45:56.139428 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:45:56.139428 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:45:56.139428 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:45:56.139428 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:45:56.139428 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:45:56.139428 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:45:56.139428 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:45:56.139428 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:45:56.139428 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:45:56.139428 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:45:56.139428 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:45:56.139428 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 633/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:45:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:45:56.144711 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37cebc-b413-4cf3-8865-5a7fa2e61ecb None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:56.244867 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37cebc-b413-4cf3-8865-5a7fa2e61ecb None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:56.273462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:56.278631 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a365115-3db5-4f45-be58-be6d30820c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:56.279256 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a365115-3db5-4f45-be58-be6d30820c4c None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:56.279532 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a365115-3db5-4f45-be58-be6d30820c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:56.279799 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a365115-3db5-4f45-be58-be6d30820c4c None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:56.280455 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a365115-3db5-4f45-be58-be6d30820c4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:45:56.280751 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a365115-3db5-4f45-be58-be6d30820c4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:56.282178 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a365115-3db5-4f45-be58-be6d30820c4c None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:56.290049 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 617/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:45:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:45:56.296969 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e99c0a-238f-4397-a18b-3cbd67c6c8c2 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:56.372192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:56.393871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e99c0a-238f-4397-a18b-3cbd67c6c8c2 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:56.426603 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed7280-3075-49d1-a1bd-d5c8a6bd5a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:56.427245 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed7280-3075-49d1-a1bd-d5c8a6bd5a11 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:56.427519 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed7280-3075-49d1-a1bd-d5c8a6bd5a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:56.427785 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed7280-3075-49d1-a1bd-d5c8a6bd5a11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:56.437545 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ed7280-3075-49d1-a1bd-d5c8a6bd5a11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:56.438995 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ed7280-3075-49d1-a1bd-d5c8a6bd5a11 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:56.453748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-29ed7280-3075-49d1-a1bd-d5c8a6bd5a11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:45:56.453748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:45:56.453748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:45:56.453748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:45:56.453748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:45:56.453748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:45:56.453748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:45:56.453748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:45:56.453748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:45:56.453748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:45:56.453748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:45:56.453748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:45:56.453748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:45:56.453748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:45:56.453748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:45:56.453748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:45:56.454849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:45:56.454849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:45:56.454849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:45:56.454849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:45:56.454849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:45:56.454849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:45:56.454849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:45:56.454849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:45:56.454849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:45:56.454849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:45:56.454849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:45:56.454849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:45:56.454849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:45:56.454849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:45:56.454849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:45:56.454849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:45:56.454849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:45:56.456127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:45:56.456127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:45:56.456127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:45:56.456127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 634/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:45:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:45:56.460737 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a365115-3db5-4f45-be58-be6d30820c4c None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:56.559461 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a365115-3db5-4f45-be58-be6d30820c4c None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:56.593037 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba30d43-7ec0-4dd0-ae87-7341c4f3285f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:56.593634 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba30d43-7ec0-4dd0-ae87-7341c4f3285f None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:56.593961 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba30d43-7ec0-4dd0-ae87-7341c4f3285f None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:56.594236 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba30d43-7ec0-4dd0-ae87-7341c4f3285f None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:56.594926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba30d43-7ec0-4dd0-ae87-7341c4f3285f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:45:56.595225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba30d43-7ec0-4dd0-ae87-7341c4f3285f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:56.596640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba30d43-7ec0-4dd0-ae87-7341c4f3285f None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:56.610501 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 618/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:45:56.616320 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed7280-3075-49d1-a1bd-d5c8a6bd5a11 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:56.713403 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed7280-3075-49d1-a1bd-d5c8a6bd5a11 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:56.739090 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:56.746850 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1b28f-991b-419f-a248-de1160d2d318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:56.747462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1b28f-991b-419f-a248-de1160d2d318 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:56.747734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1b28f-991b-419f-a248-de1160d2d318 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:56.748001 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1b28f-991b-419f-a248-de1160d2d318 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:56.755921 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab1b28f-991b-419f-a248-de1160d2d318 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:56.757415 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab1b28f-991b-419f-a248-de1160d2d318 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:56.768305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-5ab1b28f-991b-419f-a248-de1160d2d318 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 10 23:45:56.768305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:45:56.768305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:45:56.768305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:45:56.768305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:45:56.768305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:45:56.768305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:45:56.768305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:45:56.768305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:45:56.768305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:45:56.768305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:45:56.768305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:45:56.768305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:45:56.768305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:45:56.768305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:45:56.768305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:45:56.769406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:45:56.769406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:45:56.769406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:45:56.769406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:45:56.769406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:45:56.769406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:45:56.769406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:45:56.769406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:45:56.769406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:45:56.769406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:45:56.769406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:45:56.769406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:45:56.769406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:45:56.769406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:45:56.769406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:45:56.769406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:45:56.769406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:45:56.769406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:45:56.770381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:45:56.770381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:45:56.770381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:45:56.770381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:45:56.770381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:45:56.770381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:45:56.770381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:45:56.770381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:45:56.770381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 10 23:45:56.770381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:45:56.770381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 635/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:45:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:45:56.776213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba30d43-7ec0-4dd0-ae87-7341c4f3285f None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:56.832454 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:56.875609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba30d43-7ec0-4dd0-ae87-7341c4f3285f None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:56.914530 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1bae75-4176-4eea-bb03-5cf63e891fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:56.915268 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1bae75-4176-4eea-bb03-5cf63e891fbc None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:56.916210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1bae75-4176-4eea-bb03-5cf63e891fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:56.916479 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1bae75-4176-4eea-bb03-5cf63e891fbc None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:56.917197 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1bae75-4176-4eea-bb03-5cf63e891fbc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:45:56.917565 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1bae75-4176-4eea-bb03-5cf63e891fbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:56.919058 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1bae75-4176-4eea-bb03-5cf63e891fbc None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:56.936067 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 619/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Sat Sep 10 23:45:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:45:56.941871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1b28f-991b-419f-a248-de1160d2d318 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:57.041650 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1b28f-991b-419f-a248-de1160d2d318 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:57.075427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993abd6-d1dd-4ef5-9248-d443e60469f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:57.076015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993abd6-d1dd-4ef5-9248-d443e60469f3 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:57.076290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993abd6-d1dd-4ef5-9248-d443e60469f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:57.076555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993abd6-d1dd-4ef5-9248-d443e60469f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:57.131243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:57.143793 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d993abd6-d1dd-4ef5-9248-d443e60469f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:57.145276 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d993abd6-d1dd-4ef5-9248-d443e60469f3 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:57.239003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 636/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:45:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 23:45:57.245066 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1bae75-4176-4eea-bb03-5cf63e891fbc None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:57.346765 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1bae75-4176-4eea-bb03-5cf63e891fbc None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:57.360102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:57.380564 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0577d14-5751-4a72-9cf8-af6b7a4a9db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:57.381174 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0577d14-5751-4a72-9cf8-af6b7a4a9db3 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:57.381498 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0577d14-5751-4a72-9cf8-af6b7a4a9db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:57.381716 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0577d14-5751-4a72-9cf8-af6b7a4a9db3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:57.392480 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0577d14-5751-4a72-9cf8-af6b7a4a9db3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:57.393975 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0577d14-5751-4a72-9cf8-af6b7a4a9db3 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:57.409221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-d0577d14-5751-4a72-9cf8-af6b7a4a9db3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:45:57.409221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:45:57.409221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:45:57.409221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:45:57.409221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:45:57.409221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:45:57.409221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:45:57.409221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:45:57.409221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:45:57.409221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:45:57.409221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:45:57.409221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:45:57.409221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:45:57.409221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:45:57.409221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:45:57.409221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:45:57.410471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:45:57.410471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:45:57.410471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:45:57.410471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:45:57.410471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:45:57.410471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:45:57.410471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:45:57.410471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:45:57.410471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:45:57.410471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:45:57.410471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:45:57.410471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:45:57.410471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:45:57.410471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:45:57.410471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:45:57.410471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:45:57.410471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:45:57.411757 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:45:57.411757 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:45:57.411757 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:45:57.414865 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 620/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:45:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 23:45:57.416713 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993abd6-d1dd-4ef5-9248-d443e60469f3 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:57.502137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:57.514333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993abd6-d1dd-4ef5-9248-d443e60469f3 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:57.547616 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21986815-5f41-4980-8fcc-5ddaa9f12055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:57.548197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21986815-5f41-4980-8fcc-5ddaa9f12055 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:57.548465 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21986815-5f41-4980-8fcc-5ddaa9f12055 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:57.548731 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21986815-5f41-4980-8fcc-5ddaa9f12055 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:57.549352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21986815-5f41-4980-8fcc-5ddaa9f12055 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:45:57.549648 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21986815-5f41-4980-8fcc-5ddaa9f12055 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:57.551023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21986815-5f41-4980-8fcc-5ddaa9f12055 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:57.562682 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 637/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:45:57.568684 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0577d14-5751-4a72-9cf8-af6b7a4a9db3 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:57.668642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0577d14-5751-4a72-9cf8-af6b7a4a9db3 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:57.707122 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020adf5-63ac-489c-b240-19d704d76f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:57.707743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020adf5-63ac-489c-b240-19d704d76f43 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:57.708018 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020adf5-63ac-489c-b240-19d704d76f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:57.708285 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020adf5-63ac-489c-b240-19d704d76f43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:57.715620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f020adf5-63ac-489c-b240-19d704d76f43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:57.717060 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f020adf5-63ac-489c-b240-19d704d76f43 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:57.727844 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-f020adf5-63ac-489c-b240-19d704d76f43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:45:57.727844 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:45:57.727844 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:45:57.727844 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:45:57.727844 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:45:57.727844 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:45:57.727844 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:45:57.727844 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:45:57.727844 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:45:57.727844 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:45:57.727844 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:45:57.727844 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:45:57.727844 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:45:57.727844 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:45:57.727844 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:45:57.729471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:45:57.729471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:45:57.729471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:45:57.729471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:45:57.729471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:45:57.729471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:45:57.729471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:45:57.729471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:45:57.729471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:45:57.729471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:45:57.729471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:45:57.729471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:45:57.729471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:45:57.729471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:45:57.729471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:45:57.729471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:45:57.729471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:45:57.730650 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:45:57.730650 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:45:57.730650 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:45:57.730650 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:45:57.730650 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:45:57.730650 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:45:57.730650 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:45:57.730650 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:45:57.730650 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 621/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:45:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:45:57.735820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21986815-5f41-4980-8fcc-5ddaa9f12055 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:57.835845 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21986815-5f41-4980-8fcc-5ddaa9f12055 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:57.869115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1557f5-130e-4e2d-98c9-a47038f48ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:57.869707 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1557f5-130e-4e2d-98c9-a47038f48ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:57.869982 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1557f5-130e-4e2d-98c9-a47038f48ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:57.870251 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1557f5-130e-4e2d-98c9-a47038f48ae9 None admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:57.871050 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1557f5-130e-4e2d-98c9-a47038f48ae9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:45:57.871353 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1557f5-130e-4e2d-98c9-a47038f48ae9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:57.872787 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1557f5-130e-4e2d-98c9-a47038f48ae9 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:57.885294 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 638/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:45:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:45:57.891735 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020adf5-63ac-489c-b240-19d704d76f43 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:45:57.917598 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:45:57.992452 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020adf5-63ac-489c-b240-19d704d76f43 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:45:58.027014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed86b7f-2f30-4511-be1e-96fbd6e4c5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:45:58.027600 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed86b7f-2f30-4511-be1e-96fbd6e4c5e0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:45:58.027874 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed86b7f-2f30-4511-be1e-96fbd6e4c5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:45:58.028141 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed86b7f-2f30-4511-be1e-96fbd6e4c5e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:45:58.028849 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed86b7f-2f30-4511-be1e-96fbd6e4c5e0 None admin] RBAC: Adding query filter params (role.id=9dce48a2b53c482d931dcd064b07744e, scope.project.id=947277c4458645899287e6818947c076, user.id=5cb1d85a338e40f18e61887a1becb906) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:45:58.029183 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed86b7f-2f30-4511-be1e-96fbd6e4c5e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:45:58.030599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed86b7f-2f30-4511-be1e-96fbd6e4c5e0 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:45:58.036040 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 622/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:45:57 2022] GET /identity/v3/role_assignments?user.id=5cb1d85a338e40f18e61887a1becb906&scope.project.id=947277c4458645899287e6818947c076&role.id=9dce48a2b53c482d931dcd064b07744e => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:46:00.157618 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d84b2-ccf9-4d3d-827f-6713cfc156e8 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:00.158141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d84b2-ccf9-4d3d-827f-6713cfc156e8 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:00.158141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d84b2-ccf9-4d3d-827f-6713cfc156e8 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:00.159497 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 639/1261] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:46:00.164551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d86f38-1160-43ba-bc1f-dafff65acb0f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:00.164953 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d86f38-1160-43ba-bc1f-dafff65acb0f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:00.165096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d86f38-1160-43ba-bc1f-dafff65acb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:00.184125 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b6d86f38-1160-43ba-bc1f-dafff65acb0f None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:46:00.185302 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b6d86f38-1160-43ba-bc1f-dafff65acb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:46:00.252467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 623/1262] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:46:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:46:00.531454 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d84b2-ccf9-4d3d-827f-6713cfc156e8 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:00.543078 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:00.632480 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d84b2-ccf9-4d3d-827f-6713cfc156e8 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:00.672588 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce43e77-431f-44cb-a894-aed1da92b1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:00.673255 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce43e77-431f-44cb-a894-aed1da92b1af None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:00.673535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce43e77-431f-44cb-a894-aed1da92b1af None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:00.673801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce43e77-431f-44cb-a894-aed1da92b1af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:00.688851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce43e77-431f-44cb-a894-aed1da92b1af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:46:00.693582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-dce43e77-431f-44cb-a894-aed1da92b1af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:46:00.693582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:46:00.693582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:46:00.693582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:46:00.693582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:46:00.693582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:46:00.693582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:46:00.693582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:46:00.693582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:46:00.693582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:46:00.693582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:46:00.693582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:46:00.693582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:46:00.693582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:46:00.693582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:46:00.693582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:46:00.697878 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:46:00.697878 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:46:00.697878 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:46:00.697878 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:46:00.697878 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:46:00.697878 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:46:00.697878 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:46:00.697878 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:46:00.697878 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:46:00.697878 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:46:00.697878 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:46:00.697878 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:46:00.697878 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:46:00.697878 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:46:00.697878 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:46:00.697878 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:46:00.699056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:46:00.699056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:46:00.699056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:46:00.699056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:46:00.699056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:46:00.699056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:46:00.699056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:46:00.699056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:46:00.699056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:46:00.699056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:46:00.699056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:46:00.699056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:46:00.699056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 640/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:46:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:46:00.701725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d86f38-1160-43ba-bc1f-dafff65acb0f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:00.719765 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:00.797499 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d86f38-1160-43ba-bc1f-dafff65acb0f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:00.831278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22195d-2b7b-4f9e-b1a5-99fe8af1ed79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:00.831866 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22195d-2b7b-4f9e-b1a5-99fe8af1ed79 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:00.832138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22195d-2b7b-4f9e-b1a5-99fe8af1ed79 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:00.832403 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22195d-2b7b-4f9e-b1a5-99fe8af1ed79 None admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:00.833049 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22195d-2b7b-4f9e-b1a5-99fe8af1ed79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:46:00.833343 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22195d-2b7b-4f9e-b1a5-99fe8af1ed79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:00.834728 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22195d-2b7b-4f9e-b1a5-99fe8af1ed79 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:00.842795 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 624/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:46:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 23:46:00.848648 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce43e77-431f-44cb-a894-aed1da92b1af None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:00.914810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:00.947728 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce43e77-431f-44cb-a894-aed1da92b1af None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:00.981900 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79064583-e179-41fb-8cbb-df17552fc761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:00.982534 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79064583-e179-41fb-8cbb-df17552fc761 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:00.982844 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79064583-e179-41fb-8cbb-df17552fc761 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:00.983114 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79064583-e179-41fb-8cbb-df17552fc761 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:00.993986 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79064583-e179-41fb-8cbb-df17552fc761 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:00.995448 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79064583-e179-41fb-8cbb-df17552fc761 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:01.010115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-79064583-e179-41fb-8cbb-df17552fc761 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:46:01.010115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:46:01.010115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:46:01.010115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:46:01.010115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:46:01.010115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:46:01.010115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:46:01.010115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:46:01.010115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:46:01.010115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:46:01.010115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:46:01.010115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:46:01.010115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:46:01.010115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:46:01.010115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:46:01.010115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:46:01.011253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:46:01.011253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:46:01.011253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:46:01.011253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:46:01.011253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:46:01.011253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:46:01.011253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:46:01.011253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:46:01.011253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:46:01.011253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:46:01.011253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:46:01.011253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:46:01.011253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:46:01.011253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:46:01.011253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:46:01.011253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:46:01.011253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:46:01.012152 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:46:01.012152 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:46:01.012152 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:46:01.012152 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 641/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:46:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:46:01.017183 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22195d-2b7b-4f9e-b1a5-99fe8af1ed79 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:01.096154 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:01.117917 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22195d-2b7b-4f9e-b1a5-99fe8af1ed79 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:01.157166 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44098b84-de28-471a-a554-e6b8cadf69a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:01.157749 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44098b84-de28-471a-a554-e6b8cadf69a6 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:01.158021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44098b84-de28-471a-a554-e6b8cadf69a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:01.158285 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44098b84-de28-471a-a554-e6b8cadf69a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:01.158936 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44098b84-de28-471a-a554-e6b8cadf69a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:46:01.159230 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44098b84-de28-471a-a554-e6b8cadf69a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:01.160576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44098b84-de28-471a-a554-e6b8cadf69a6 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:01.171790 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 625/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:46:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:46:01.177638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79064583-e179-41fb-8cbb-df17552fc761 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:01.276105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79064583-e179-41fb-8cbb-df17552fc761 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:01.294727 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:01.309710 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f07ada-939a-43e6-9ba1-3ba0620d471f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:01.310376 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f07ada-939a-43e6-9ba1-3ba0620d471f None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:01.310644 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f07ada-939a-43e6-9ba1-3ba0620d471f None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:01.310931 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f07ada-939a-43e6-9ba1-3ba0620d471f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:01.318317 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f07ada-939a-43e6-9ba1-3ba0620d471f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:01.319781 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f07ada-939a-43e6-9ba1-3ba0620d471f None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:01.331047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-31f07ada-939a-43e6-9ba1-3ba0620d471f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 10 23:46:01.331047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:46:01.331047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:46:01.331047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:46:01.331047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:46:01.331047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:46:01.331047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:46:01.331047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:46:01.331047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:46:01.331047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:46:01.331047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:46:01.331047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:46:01.331047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:46:01.331047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:46:01.331047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:46:01.331047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:46:01.332132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:46:01.332132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:46:01.332132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:46:01.332132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:46:01.332132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:46:01.332132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:46:01.332132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:46:01.332132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:46:01.332132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:46:01.332132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:46:01.332132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:46:01.332132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:46:01.332132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:46:01.332132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:46:01.332132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:46:01.332132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:46:01.332132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:46:01.332132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:46:01.333577 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:46:01.333577 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:46:01.333577 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:46:01.333577 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:46:01.333577 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:46:01.333577 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:46:01.333577 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:46:01.333577 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:46:01.333577 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 10 23:46:01.333577 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:46:01.333577 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 642/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:46:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:46:01.338176 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44098b84-de28-471a-a554-e6b8cadf69a6 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:01.434126 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44098b84-de28-471a-a554-e6b8cadf69a6 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:01.469436 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb6b19-4b20-44d7-8fef-e00d3571ea66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:01.470023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb6b19-4b20-44d7-8fef-e00d3571ea66 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:01.470294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb6b19-4b20-44d7-8fef-e00d3571ea66 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:01.470558 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb6b19-4b20-44d7-8fef-e00d3571ea66 None admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:01.471289 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb6b19-4b20-44d7-8fef-e00d3571ea66 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:46:01.471584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb6b19-4b20-44d7-8fef-e00d3571ea66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:01.472982 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb6b19-4b20-44d7-8fef-e00d3571ea66 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:01.482551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:01.490258 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 626/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Sat Sep 10 23:46:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:46:01.496108 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f07ada-939a-43e6-9ba1-3ba0620d471f None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:01.594513 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f07ada-939a-43e6-9ba1-3ba0620d471f None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:01.627769 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5b00a-c7e1-4a23-9a32-1594d96c064d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:01.628386 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5b00a-c7e1-4a23-9a32-1594d96c064d None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:01.628666 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5b00a-c7e1-4a23-9a32-1594d96c064d None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:01.628933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5b00a-c7e1-4a23-9a32-1594d96c064d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:01.639817 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f5b00a-c7e1-4a23-9a32-1594d96c064d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:01.641235 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f5b00a-c7e1-4a23-9a32-1594d96c064d None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:01.655501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-55f5b00a-c7e1-4a23-9a32-1594d96c064d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:46:01.655501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:46:01.655501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:46:01.655501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:46:01.655501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:46:01.655501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:46:01.655501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:46:01.655501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:46:01.655501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:46:01.655501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:46:01.655501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:46:01.655501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:46:01.655501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:46:01.655501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:46:01.655501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:46:01.655501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:46:01.656598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:46:01.656598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:46:01.656598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:46:01.656598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:46:01.656598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:46:01.656598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:46:01.656598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:46:01.656598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:46:01.656598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:46:01.656598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:46:01.656598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:46:01.656598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:46:01.656598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:46:01.656598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:46:01.656598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:46:01.656598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:46:01.656598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:46:01.657669 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:46:01.657669 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:46:01.657669 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:46:01.657669 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 643/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:46:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:46:01.662323 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb6b19-4b20-44d7-8fef-e00d3571ea66 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:01.758860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb6b19-4b20-44d7-8fef-e00d3571ea66 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:01.792602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e620d01-b0bb-45d1-b5e5-e49cf68879c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:01.793189 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e620d01-b0bb-45d1-b5e5-e49cf68879c4 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:01.793462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e620d01-b0bb-45d1-b5e5-e49cf68879c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:01.793728 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e620d01-b0bb-45d1-b5e5-e49cf68879c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:01.794350 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e620d01-b0bb-45d1-b5e5-e49cf68879c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:46:01.794643 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e620d01-b0bb-45d1-b5e5-e49cf68879c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:01.796039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e620d01-b0bb-45d1-b5e5-e49cf68879c4 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:01.806974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 627/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:46:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:46:01.812915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5b00a-c7e1-4a23-9a32-1594d96c064d None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:01.844537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:01.909515 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5b00a-c7e1-4a23-9a32-1594d96c064d None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:01.943116 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf8cd5-0008-4ce1-8a22-97b40953af39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:01.943731 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf8cd5-0008-4ce1-8a22-97b40953af39 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:01.944005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf8cd5-0008-4ce1-8a22-97b40953af39 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:01.944272 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf8cd5-0008-4ce1-8a22-97b40953af39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:01.951133 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf8cd5-0008-4ce1-8a22-97b40953af39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:01.952507 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf8cd5-0008-4ce1-8a22-97b40953af39 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:01.964992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-00bf8cd5-0008-4ce1-8a22-97b40953af39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:46:01.964992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:46:01.964992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:46:01.964992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:46:01.964992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:46:01.964992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:46:01.964992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:46:01.964992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:46:01.964992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:46:01.964992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:46:01.964992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:46:01.964992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:46:01.964992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:46:01.964992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:46:01.964992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:46:01.966055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:46:01.966055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:46:01.966055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:46:01.966055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:46:01.966055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:46:01.966055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:46:01.966055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:46:01.966055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:46:01.966055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:46:01.966055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:46:01.966055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:46:01.966055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:46:01.966055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:46:01.966055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:46:01.966055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:46:01.966055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:46:01.966055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:46:01.967112 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:46:01.967112 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:46:01.967112 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:46:01.967112 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:46:01.967112 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:46:01.967112 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:46:01.967112 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:46:01.967112 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:46:01.967112 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 644/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:46:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:46:01.971986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e620d01-b0bb-45d1-b5e5-e49cf68879c4 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:02.030795 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:02.071781 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e620d01-b0bb-45d1-b5e5-e49cf68879c4 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:02.104706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc98ae-d552-492d-8157-a2627a6e8de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:02.105296 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc98ae-d552-492d-8157-a2627a6e8de9 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:02.105574 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc98ae-d552-492d-8157-a2627a6e8de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:02.105843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc98ae-d552-492d-8157-a2627a6e8de9 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:02.106520 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bc98ae-d552-492d-8157-a2627a6e8de9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:46:02.106850 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bc98ae-d552-492d-8157-a2627a6e8de9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:02.108273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bc98ae-d552-492d-8157-a2627a6e8de9 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:02.119914 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 628/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:46:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:46:02.125897 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf8cd5-0008-4ce1-8a22-97b40953af39 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:02.218306 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:02.226724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf8cd5-0008-4ce1-8a22-97b40953af39 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:02.260476 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb31552-64a9-4b8f-817a-75aef35c687b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:02.261111 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb31552-64a9-4b8f-817a-75aef35c687b None admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:02.261386 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb31552-64a9-4b8f-817a-75aef35c687b None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:02.261656 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb31552-64a9-4b8f-817a-75aef35c687b None admin] PATH_INFO: `/v3/projects/947277c4458645899287e6818947c076/users/5cb1d85a338e40f18e61887a1becb906/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:02.263533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb31552-64a9-4b8f-817a-75aef35c687b None admin] RBAC: Authorizing `identity:create_grant(project_id=947277c4458645899287e6818947c076, user_id=5cb1d85a338e40f18e61887a1becb906, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:02.264937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb31552-64a9-4b8f-817a-75aef35c687b None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:02.343611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 645/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Sat Sep 10 23:46:02 2022] PUT /identity/v3/projects/947277c4458645899287e6818947c076/users/5cb1d85a338e40f18e61887a1becb906/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:46:04.505870 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d1a9b-2810-44ed-9f98-d9674e2b2bf1 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:04.506396 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d1a9b-2810-44ed-9f98-d9674e2b2bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:04.506396 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d1a9b-2810-44ed-9f98-d9674e2b2bf1 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:04.507798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 629/1274] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:46:04.513316 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78463ba4-ef7d-4a9f-8726-0dafeb8664ae None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:04.513720 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78463ba4-ef7d-4a9f-8726-0dafeb8664ae None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:04.513849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78463ba4-ef7d-4a9f-8726-0dafeb8664ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:04.532400 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-78463ba4-ef7d-4a9f-8726-0dafeb8664ae None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:46:04.533568 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-78463ba4-ef7d-4a9f-8726-0dafeb8664ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:46:04.600818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 646/1275] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:46:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:46:04.882140 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d1a9b-2810-44ed-9f98-d9674e2b2bf1 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:04.989905 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d1a9b-2810-44ed-9f98-d9674e2b2bf1 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:05.008163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:05.025479 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb357a-ad36-4e63-9d3e-2aedc9600834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:05.026110 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb357a-ad36-4e63-9d3e-2aedc9600834 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:05.026381 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb357a-ad36-4e63-9d3e-2aedc9600834 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:05.026650 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb357a-ad36-4e63-9d3e-2aedc9600834 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:05.036874 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb357a-ad36-4e63-9d3e-2aedc9600834 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 23:46:05.041843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-2bfb357a-ad36-4e63-9d3e-2aedc9600834 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:46:05.041843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:46:05.041843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:46:05.041843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:46:05.041843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:46:05.041843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:46:05.041843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:46:05.041843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:46:05.041843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:46:05.041843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:46:05.041843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:46:05.041843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:46:05.041843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 23:46:05.041843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 23:46:05.041843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 23:46:05.041843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 23:46:05.042860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 23:46:05.042860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 23:46:05.042860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 23:46:05.042860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 23:46:05.042860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:46:05.042860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:46:05.042860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:46:05.042860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:46:05.042860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:46:05.042860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:46:05.042860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:46:05.042860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:46:05.042860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:46:05.042860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:46:05.042860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:46:05.042860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:46:05.043908 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:46:05.043908 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:46:05.043908 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:46:05.043908 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:46:05.043908 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 23:46:05.043908 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 23:46:05.043908 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 23:46:05.043908 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 23:46:05.043908 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 23:46:05.043908 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 23:46:05.043908 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 23:46:05.043908 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:46:05.043908 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 630/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Sat Sep 10 23:46:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:46:05.049218 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78463ba4-ef7d-4a9f-8726-0dafeb8664ae None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:05.133144 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:05.151299 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78463ba4-ef7d-4a9f-8726-0dafeb8664ae None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:05.183942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b46e1-2e45-45e0-b392-824f836ef41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:05.184543 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b46e1-2e45-45e0-b392-824f836ef41c None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:05.184816 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b46e1-2e45-45e0-b392-824f836ef41c None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:05.185143 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b46e1-2e45-45e0-b392-824f836ef41c None admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:05.185806 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b46e1-2e45-45e0-b392-824f836ef41c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:46:05.186284 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b46e1-2e45-45e0-b392-824f836ef41c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:05.188328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b46e1-2e45-45e0-b392-824f836ef41c None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:05.196114 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 647/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Sat Sep 10 23:46:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:46:05.202848 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb357a-ad36-4e63-9d3e-2aedc9600834 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:05.303410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb357a-ad36-4e63-9d3e-2aedc9600834 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:05.337494 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d714e5-e3f6-4b4a-8e83-1863fa36ad58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:05.338098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d714e5-e3f6-4b4a-8e83-1863fa36ad58 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:05.338367 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d714e5-e3f6-4b4a-8e83-1863fa36ad58 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:05.338631 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d714e5-e3f6-4b4a-8e83-1863fa36ad58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:05.348269 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d714e5-e3f6-4b4a-8e83-1863fa36ad58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:05.349701 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d714e5-e3f6-4b4a-8e83-1863fa36ad58 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:05.364416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-87d714e5-e3f6-4b4a-8e83-1863fa36ad58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:46:05.364416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:46:05.364416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:46:05.364416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:46:05.364416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:46:05.364416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:46:05.364416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:46:05.364416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:46:05.364416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:46:05.364416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:46:05.364416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:46:05.364416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:46:05.364416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:46:05.364416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:46:05.364416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:46:05.364416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:46:05.369988 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:46:05.369988 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:46:05.369988 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:46:05.369988 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:46:05.369988 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:46:05.369988 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:46:05.369988 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:46:05.369988 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:46:05.369988 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:46:05.369988 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:46:05.369988 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:46:05.369988 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:46:05.369988 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:46:05.369988 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:46:05.369988 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:46:05.369988 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:46:05.369988 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:46:05.371120 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:46:05.371120 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:46:05.371120 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:46:05.371120 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 631/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:46:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:46:05.372131 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b46e1-2e45-45e0-b392-824f836ef41c None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:05.468877 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b46e1-2e45-45e0-b392-824f836ef41c None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:05.502046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718d027-ac64-4748-87a9-c9bc87ed3fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:05.502652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718d027-ac64-4748-87a9-c9bc87ed3fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:05.502952 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718d027-ac64-4748-87a9-c9bc87ed3fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:05.503218 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718d027-ac64-4748-87a9-c9bc87ed3fd1 None admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:05.503848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8718d027-ac64-4748-87a9-c9bc87ed3fd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:46:05.504146 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8718d027-ac64-4748-87a9-c9bc87ed3fd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:05.505492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8718d027-ac64-4748-87a9-c9bc87ed3fd1 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:05.517384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 648/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:46:05.523242 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d714e5-e3f6-4b4a-8e83-1863fa36ad58 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:05.554022 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:05.622757 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d714e5-e3f6-4b4a-8e83-1863fa36ad58 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:05.655519 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf5df6-db0c-408f-998e-73ed92dddf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:05.656128 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf5df6-db0c-408f-998e-73ed92dddf94 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:05.656401 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf5df6-db0c-408f-998e-73ed92dddf94 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:05.656664 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf5df6-db0c-408f-998e-73ed92dddf94 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:05.663533 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf5df6-db0c-408f-998e-73ed92dddf94 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:05.664962 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf5df6-db0c-408f-998e-73ed92dddf94 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:05.676620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-eaaf5df6-db0c-408f-998e-73ed92dddf94 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 10 23:46:05.676620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:46:05.676620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:46:05.676620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:46:05.676620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:46:05.676620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:46:05.676620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:46:05.676620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:46:05.676620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:46:05.676620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:46:05.676620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:46:05.676620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:46:05.676620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 23:46:05.676620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 23:46:05.676620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 23:46:05.676620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 23:46:05.677944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:46:05.677944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:46:05.677944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 23:46:05.677944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:46:05.677944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 23:46:05.677944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 23:46:05.677944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:46:05.677944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:46:05.677944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:46:05.677944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:46:05.677944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:46:05.677944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:46:05.677944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:46:05.677944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:46:05.677944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:46:05.677944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:46:05.677944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:46:05.677944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:46:05.683945 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:46:05.683945 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:46:05.683945 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 23:46:05.683945 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 23:46:05.683945 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 23:46:05.683945 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 23:46:05.683945 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 23:46:05.683945 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 23:46:05.683945 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 10 23:46:05.683945 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:46:05.683945 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 632/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:46:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:46:05.684725 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718d027-ac64-4748-87a9-c9bc87ed3fd1 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:05.688410 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:05.795380 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718d027-ac64-4748-87a9-c9bc87ed3fd1 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:05.832460 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ab06a-10af-47cf-bf62-2442ede7d484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:05.833064 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ab06a-10af-47cf-bf62-2442ede7d484 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:05.833340 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ab06a-10af-47cf-bf62-2442ede7d484 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:05.833735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ab06a-10af-47cf-bf62-2442ede7d484 None admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:05.834461 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3ab06a-10af-47cf-bf62-2442ede7d484 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:46:05.834852 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3ab06a-10af-47cf-bf62-2442ede7d484 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:05.836313 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3ab06a-10af-47cf-bf62-2442ede7d484 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:05.855366 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 649/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Sat Sep 10 23:46:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:46:05.861371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf5df6-db0c-408f-998e-73ed92dddf94 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:05.956557 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:05.966400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf5df6-db0c-408f-998e-73ed92dddf94 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:06.000032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d14bba-a29c-4eae-afbb-a626c0d3bd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:06.000618 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d14bba-a29c-4eae-afbb-a626c0d3bd32 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:06.000892 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d14bba-a29c-4eae-afbb-a626c0d3bd32 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:06.001156 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d14bba-a29c-4eae-afbb-a626c0d3bd32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:06.067512 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d14bba-a29c-4eae-afbb-a626c0d3bd32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:06.068886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d14bba-a29c-4eae-afbb-a626c0d3bd32 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:06.155199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:06.164961 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 633/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Sat Sep 10 23:46:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 23:46:06.171186 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ab06a-10af-47cf-bf62-2442ede7d484 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:06.231872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:06.269163 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ab06a-10af-47cf-bf62-2442ede7d484 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:06.306909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821cb2b-d0f7-4009-b11b-9c60208ddbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:06.307561 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821cb2b-d0f7-4009-b11b-9c60208ddbdd None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:06.307864 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821cb2b-d0f7-4009-b11b-9c60208ddbdd None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:06.308127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821cb2b-d0f7-4009-b11b-9c60208ddbdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:06.318145 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821cb2b-d0f7-4009-b11b-9c60208ddbdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:06.319529 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821cb2b-d0f7-4009-b11b-9c60208ddbdd None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:06.333987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-e821cb2b-d0f7-4009-b11b-9c60208ddbdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:46:06.333987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:46:06.333987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:46:06.333987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:46:06.333987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:46:06.333987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:46:06.333987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:46:06.333987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:46:06.333987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:46:06.333987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:46:06.333987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:46:06.333987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:46:06.333987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 23:46:06.333987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 23:46:06.333987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 23:46:06.333987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 23:46:06.338872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:46:06.338872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:46:06.338872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:46:06.338872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:46:06.338872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:46:06.338872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:46:06.338872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:46:06.338872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:46:06.338872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:46:06.338872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:46:06.338872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:46:06.338872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:46:06.338872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:46:06.338872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:46:06.338872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:46:06.338872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:46:06.338872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 23:46:06.339969 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 23:46:06.339969 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 23:46:06.339969 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:46:06.339969 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 650/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:46:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:46:06.341845 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d14bba-a29c-4eae-afbb-a626c0d3bd32 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:06.436783 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d14bba-a29c-4eae-afbb-a626c0d3bd32 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:06.469113 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0e7567-958e-46e1-ae9c-18d62939b4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:06.469696 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e7567-958e-46e1-ae9c-18d62939b4cc None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:06.469969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e7567-958e-46e1-ae9c-18d62939b4cc None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:06.470235 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e7567-958e-46e1-ae9c-18d62939b4cc None admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:06.470881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0e7567-958e-46e1-ae9c-18d62939b4cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:46:06.471177 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0e7567-958e-46e1-ae9c-18d62939b4cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:06.479118 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0e7567-958e-46e1-ae9c-18d62939b4cc None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:06.483265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 634/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Sat Sep 10 23:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:46:06.489212 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821cb2b-d0f7-4009-b11b-9c60208ddbdd None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:06.586245 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821cb2b-d0f7-4009-b11b-9c60208ddbdd None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:06.599152 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:06.618844 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208dc85-1a13-497b-bde3-163ae4c84cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:06.619867 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208dc85-1a13-497b-bde3-163ae4c84cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:06.620146 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208dc85-1a13-497b-bde3-163ae4c84cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:06.620420 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208dc85-1a13-497b-bde3-163ae4c84cc9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:06.626775 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3208dc85-1a13-497b-bde3-163ae4c84cc9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:06.629861 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3208dc85-1a13-497b-bde3-163ae4c84cc9 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:06.640072 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application [None req-3208dc85-1a13-497b-bde3-163ae4c84cc9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:46:06.640072 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:46:06.640072 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:46:06.640072 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:46:06.640072 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:46:06.640072 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:46:06.640072 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:46:06.640072 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:46:06.640072 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:46:06.640072 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:46:06.640072 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:46:06.640072 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:46:06.640072 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 23:46:06.640072 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 23:46:06.640072 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 23:46:06.641042 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 23:46:06.641042 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:46:06.641042 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:46:06.641042 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:46:06.641042 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:46:06.641042 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:46:06.641042 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:46:06.641042 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:46:06.641042 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application with Lock( Sep 10 23:46:06.641042 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:46:06.641042 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:46:06.641042 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:46:06.641042 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:46:06.641042 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:46:06.641042 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:46:06.641042 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:46:06.641042 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:46:06.642334 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 23:46:06.642334 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 23:46:06.642334 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 23:46:06.642334 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 23:46:06.642334 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 23:46:06.642334 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 23:46:06.642334 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 23:46:06.642334 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: ERROR keystone.server.flask.application Sep 10 23:46:06.642334 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 651/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Sat Sep 10 23:46:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:46:06.647111 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0e7567-958e-46e1-ae9c-18d62939b4cc None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:06.695655 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:06.746085 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0e7567-958e-46e1-ae9c-18d62939b4cc None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:06.779639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf97d2-e161-46c5-b3f9-2c781bbac103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:06.780250 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf97d2-e161-46c5-b3f9-2c781bbac103 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:06.780523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf97d2-e161-46c5-b3f9-2c781bbac103 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:06.780791 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf97d2-e161-46c5-b3f9-2c781bbac103 None admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:06.781477 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf97d2-e161-46c5-b3f9-2c781bbac103 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:46:06.781778 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf97d2-e161-46c5-b3f9-2c781bbac103 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:06.783234 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf97d2-e161-46c5-b3f9-2c781bbac103 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:06.795262 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 635/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Sat Sep 10 23:46:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:46:06.801379 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208dc85-1a13-497b-bde3-163ae4c84cc9 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:06.901821 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208dc85-1a13-497b-bde3-163ae4c84cc9 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:06.936307 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee05a2-d464-4a6b-99cf-a192fc293906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:06.936917 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee05a2-d464-4a6b-99cf-a192fc293906 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:06.937167 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee05a2-d464-4a6b-99cf-a192fc293906 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:06.937435 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee05a2-d464-4a6b-99cf-a192fc293906 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:06.938141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ee05a2-d464-4a6b-99cf-a192fc293906 None admin] RBAC: Adding query filter params (role.id=9dce48a2b53c482d931dcd064b07744e, scope.project.id=947277c4458645899287e6818947c076, user.id=5cb1d85a338e40f18e61887a1becb906) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:46:06.938442 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ee05a2-d464-4a6b-99cf-a192fc293906 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:06.939877 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ee05a2-d464-4a6b-99cf-a192fc293906 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:06.945440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 652/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Sat Sep 10 23:46:06 2022] GET /identity/v3/role_assignments?user.id=5cb1d85a338e40f18e61887a1becb906&scope.project.id=947277c4458645899287e6818947c076&role.id=9dce48a2b53c482d931dcd064b07744e => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:46:09.196201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda437fe-95f0-46d3-8b75-ee3fb3c87eb6 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:09.197064 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda437fe-95f0-46d3-8b75-ee3fb3c87eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:09.197064 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda437fe-95f0-46d3-8b75-ee3fb3c87eb6 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:09.198096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 636/1288] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:46:09.203797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d8225-eea6-4d8f-8e0a-140df93ac163 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:09.204273 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d8225-eea6-4d8f-8e0a-140df93ac163 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:09.204373 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d8225-eea6-4d8f-8e0a-140df93ac163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:09.225116 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-4b2d8225-eea6-4d8f-8e0a-140df93ac163 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:46:09.226308 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4b2d8225-eea6-4d8f-8e0a-140df93ac163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:46:09.259544 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:09.311683 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 653/1289] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:46:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:46:09.603951 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda437fe-95f0-46d3-8b75-ee3fb3c87eb6 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:09.703846 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda437fe-95f0-46d3-8b75-ee3fb3c87eb6 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:09.715079 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:09.738210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d19937-aa8d-4082-873c-c7f0f07acbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:09.738857 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d19937-aa8d-4082-873c-c7f0f07acbae None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:09.739139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d19937-aa8d-4082-873c-c7f0f07acbae None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:09.739410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d19937-aa8d-4082-873c-c7f0f07acbae None admin] PATH_INFO: `/v3/services/placement` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:09.739989 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d19937-aa8d-4082-873c-c7f0f07acbae None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:09.741355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d19937-aa8d-4082-873c-c7f0f07acbae None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:09.750216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-e4d19937-aa8d-4082-873c-c7f0f07acbae None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 23:46:09.750216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:46:09.750216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 23:46:09.750216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 23:46:09.750216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 23:46:09.750216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 23:46:09.750216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 23:46:09.750216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:46:09.750216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 23:46:09.750216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:46:09.750216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:46:09.750216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:46:09.750216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:46:09.750216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:46:09.750216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:46:09.750216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:46:09.750216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:46:09.751421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:46:09.751421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:46:09.751421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:46:09.751421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:46:09.751421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:46:09.751421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:46:09.751421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 23:46:09.751421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 23:46:09.751421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 23:46:09.751421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 23:46:09.751421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:46:09.751421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:46:09.751421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:46:09.751421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:46:09.751421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:46:09.751421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:46:09.752433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:46:09.752433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:46:09.752433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:46:09.752433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:46:09.752433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:46:09.752433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:46:09.752433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:46:09.752433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:46:09.752433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:46:09.752433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:46:09.752433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 23:46:09.752433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:46:09.752433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 23:46:09.752433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:46:09.752433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 637/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Sep 10 23:46:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:46:09.757482 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d8225-eea6-4d8f-8e0a-140df93ac163 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:09.856995 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d8225-eea6-4d8f-8e0a-140df93ac163 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:09.891177 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71c32d-81a1-45b0-b3b9-b97cc4281cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:09.891777 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71c32d-81a1-45b0-b3b9-b97cc4281cda None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:09.892048 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71c32d-81a1-45b0-b3b9-b97cc4281cda None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:09.892355 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71c32d-81a1-45b0-b3b9-b97cc4281cda None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:09.893008 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d71c32d-81a1-45b0-b3b9-b97cc4281cda None admin] RBAC: Adding query filter params (name=placement) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:46:09.893304 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d71c32d-81a1-45b0-b3b9-b97cc4281cda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:09.894650 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d71c32d-81a1-45b0-b3b9-b97cc4281cda None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:09.900210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 654/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Sat Sep 10 23:46:09 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:46:09.906046 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d19937-aa8d-4082-873c-c7f0f07acbae None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:10.005213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d19937-aa8d-4082-873c-c7f0f07acbae None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:10.039250 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6a627-56c2-4625-89ac-638a86e06767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:10.039836 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6a627-56c2-4625-89ac-638a86e06767 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:10.040138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6a627-56c2-4625-89ac-638a86e06767 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:10.040407 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6a627-56c2-4625-89ac-638a86e06767 None admin] PATH_INFO: `/v3/services` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:10.041065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6a627-56c2-4625-89ac-638a86e06767 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:46:10.041364 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6a627-56c2-4625-89ac-638a86e06767 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:10.042759 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6a627-56c2-4625-89ac-638a86e06767 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:10.047060 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 638/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Sat Sep 10 23:46:09 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:46:12.191676 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36d428-307c-4c28-8f00-97351d13cf13 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:12.192224 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36d428-307c-4c28-8f00-97351d13cf13 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:12.192224 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36d428-307c-4c28-8f00-97351d13cf13 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:12.193537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 655/1293] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:46:12.199246 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57e94d-0cdf-4aa5-8f80-115e4981b1df None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:12.199686 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57e94d-0cdf-4aa5-8f80-115e4981b1df None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:12.199794 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57e94d-0cdf-4aa5-8f80-115e4981b1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:12.218458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1f57e94d-0cdf-4aa5-8f80-115e4981b1df None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:46:12.219671 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1f57e94d-0cdf-4aa5-8f80-115e4981b1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:46:12.247634 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:12.283135 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 639/1294] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:46:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 23:46:12.555900 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36d428-307c-4c28-8f00-97351d13cf13 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:12.586189 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:12.654397 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36d428-307c-4c28-8f00-97351d13cf13 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:12.687635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31c959-a66d-46de-8881-d8f86959dfd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:12.688219 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31c959-a66d-46de-8881-d8f86959dfd9 None admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:12.688488 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31c959-a66d-46de-8881-d8f86959dfd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:12.688748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31c959-a66d-46de-8881-d8f86959dfd9 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:12.689384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a31c959-a66d-46de-8881-d8f86959dfd9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:12.690749 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a31c959-a66d-46de-8881-d8f86959dfd9 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:12.810250 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 656/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Sat Sep 10 23:46:12 2022] POST /identity/v3/services => generated 246 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 23:46:14.966033 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d1b70-aa2f-4709-ac01-324b02d74ea8 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:14.966592 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d1b70-aa2f-4709-ac01-324b02d74ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:14.966592 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d1b70-aa2f-4709-ac01-324b02d74ea8 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:14.967980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 640/1296] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:46:14.973264 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba76f3-cc3c-4c07-9cd3-92559da4d9cc None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:14.973677 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba76f3-cc3c-4c07-9cd3-92559da4d9cc None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:14.973778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba76f3-cc3c-4c07-9cd3-92559da4d9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:14.992045 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-3eba76f3-cc3c-4c07-9cd3-92559da4d9cc None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:46:14.995411 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3eba76f3-cc3c-4c07-9cd3-92559da4d9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:46:15.064447 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 657/1297] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:46:14 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:46:15.335192 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d1b70-aa2f-4709-ac01-324b02d74ea8 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:15.438022 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d1b70-aa2f-4709-ac01-324b02d74ea8 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:15.471643 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29940c87-2e31-4eae-8293-ca089f9457f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:15.472248 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29940c87-2e31-4eae-8293-ca089f9457f0 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:15.472527 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29940c87-2e31-4eae-8293-ca089f9457f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:15.472792 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29940c87-2e31-4eae-8293-ca089f9457f0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:15.473362 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29940c87-2e31-4eae-8293-ca089f9457f0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:15.474739 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29940c87-2e31-4eae-8293-ca089f9457f0 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:15.483966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-29940c87-2e31-4eae-8293-ca089f9457f0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 23:46:15.483966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:46:15.483966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 23:46:15.483966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 23:46:15.483966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 23:46:15.483966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 23:46:15.483966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 23:46:15.483966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:46:15.483966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 23:46:15.483966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:46:15.483966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:46:15.483966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:46:15.483966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:46:15.483966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:46:15.483966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:46:15.483966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:46:15.483966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:46:15.485096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:46:15.485096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:46:15.485096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:46:15.485096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:46:15.485096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:46:15.485096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:46:15.485096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 23:46:15.485096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 23:46:15.485096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 23:46:15.485096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 23:46:15.485096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:46:15.485096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:46:15.485096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:46:15.485096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:46:15.485096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:46:15.485096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:46:15.486045 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:46:15.486045 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:46:15.486045 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:46:15.486045 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:46:15.486045 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:46:15.486045 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:46:15.486045 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:46:15.486045 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:46:15.486045 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:46:15.486045 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:46:15.486045 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 23:46:15.486045 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:46:15.486045 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 23:46:15.486045 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:46:15.486045 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 641/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Sep 10 23:46:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:46:15.491114 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba76f3-cc3c-4c07-9cd3-92559da4d9cc None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:15.513219 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:15.590720 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba76f3-cc3c-4c07-9cd3-92559da4d9cc None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:15.624298 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccfc1ce-3fc7-41a1-846f-0c0a2d9a36a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:15.626521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfc1ce-3fc7-41a1-846f-0c0a2d9a36a5 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:15.626840 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfc1ce-3fc7-41a1-846f-0c0a2d9a36a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:15.627119 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfc1ce-3fc7-41a1-846f-0c0a2d9a36a5 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:15.627750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccfc1ce-3fc7-41a1-846f-0c0a2d9a36a5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:46:15.628058 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccfc1ce-3fc7-41a1-846f-0c0a2d9a36a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:15.629424 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccfc1ce-3fc7-41a1-846f-0c0a2d9a36a5 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:15.640650 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 658/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Sat Sep 10 23:46:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:46:15.646647 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29940c87-2e31-4eae-8293-ca089f9457f0 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:15.677586 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:15.744964 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29940c87-2e31-4eae-8293-ca089f9457f0 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:15.777528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f5caa1-0ebf-4d8f-8782-74b3eb39b92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:15.778107 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5caa1-0ebf-4d8f-8782-74b3eb39b92b None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:15.778378 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5caa1-0ebf-4d8f-8782-74b3eb39b92b None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:15.778638 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5caa1-0ebf-4d8f-8782-74b3eb39b92b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:15.779396 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f5caa1-0ebf-4d8f-8782-74b3eb39b92b None admin] RBAC: Adding query filter params (interface=public, service_id=4a0e2afa5d244a4680275ccf8923535f, region_id=RegionOne) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:46:15.779694 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f5caa1-0ebf-4d8f-8782-74b3eb39b92b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:15.781049 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f5caa1-0ebf-4d8f-8782-74b3eb39b92b None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:15.785886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 642/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Sat Sep 10 23:46:15 2022] GET /identity/v3/endpoints?service_id=4a0e2afa5d244a4680275ccf8923535f&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:46:15.792241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccfc1ce-3fc7-41a1-846f-0c0a2d9a36a5 None admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:15.875953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:15.893055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccfc1ce-3fc7-41a1-846f-0c0a2d9a36a5 None admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:15.926524 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e082f50a-02c3-496d-8db1-52648aada350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:15.932647 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e082f50a-02c3-496d-8db1-52648aada350 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:15.932647 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e082f50a-02c3-496d-8db1-52648aada350 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:15.932647 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e082f50a-02c3-496d-8db1-52648aada350 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:15.932647 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e082f50a-02c3-496d-8db1-52648aada350 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:15.932647 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e082f50a-02c3-496d-8db1-52648aada350 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:15.937098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 659/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Sat Sep 10 23:46:15 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 23:46:18.113694 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d004dde-c63e-4e29-a49b-bf44e28848c0 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:18.114253 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d004dde-c63e-4e29-a49b-bf44e28848c0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:18.114253 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d004dde-c63e-4e29-a49b-bf44e28848c0 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:18.115595 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 643/1302] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:46:18.121413 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13816a-bdc1-4b10-87d8-feda6da98143 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:18.121936 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13816a-bdc1-4b10-87d8-feda6da98143 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:18.122037 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13816a-bdc1-4b10-87d8-feda6da98143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:18.140208 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-af13816a-bdc1-4b10-87d8-feda6da98143 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:46:18.141384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-af13816a-bdc1-4b10-87d8-feda6da98143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:46:18.206888 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 660/1303] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:46:18 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 23:46:18.484750 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d004dde-c63e-4e29-a49b-bf44e28848c0 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:18.577811 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:18.589158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d004dde-c63e-4e29-a49b-bf44e28848c0 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:18.628186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989e4d3-7f30-4771-a84a-51a0219d8511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:18.628812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989e4d3-7f30-4771-a84a-51a0219d8511 None admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:18.629081 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989e4d3-7f30-4771-a84a-51a0219d8511 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:18.629349 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989e4d3-7f30-4771-a84a-51a0219d8511 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:18.629931 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7989e4d3-7f30-4771-a84a-51a0219d8511 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:18.631332 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7989e4d3-7f30-4771-a84a-51a0219d8511 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:18.640096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application [None req-7989e4d3-7f30-4771-a84a-51a0219d8511 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 23:46:18.640096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:46:18.640096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 23:46:18.640096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 23:46:18.640096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 23:46:18.640096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 23:46:18.640096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 23:46:18.640096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:46:18.640096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 23:46:18.640096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:46:18.640096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 23:46:18.640096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:46:18.640096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 23:46:18.640096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 23:46:18.640096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 23:46:18.640096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 23:46:18.640096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 23:46:18.641294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 23:46:18.641294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 23:46:18.641294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 23:46:18.641294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 23:46:18.641294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 23:46:18.641294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 23:46:18.641294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 23:46:18.641294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 23:46:18.641294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 23:46:18.641294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 23:46:18.641294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 23:46:18.641294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 23:46:18.641294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 23:46:18.641294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 23:46:18.641294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 23:46:18.641294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 23:46:18.642335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 23:46:18.642335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application with Lock( Sep 10 23:46:18.642335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 23:46:18.642335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self._enter() Sep 10 23:46:18.642335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 23:46:18.642335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 23:46:18.642335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 23:46:18.642335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application return self.creator() Sep 10 23:46:18.642335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 23:46:18.642335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application created_value = creator( Sep 10 23:46:18.642335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 23:46:18.642335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 23:46:18.642335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 23:46:18.642335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: ERROR keystone.server.flask.application Sep 10 23:46:18.642335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 644/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Sat Sep 10 23:46:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 23:46:18.647391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13816a-bdc1-4b10-87d8-feda6da98143 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:18.721179 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:18.743066 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13816a-bdc1-4b10-87d8-feda6da98143 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:18.781713 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4125c-8953-4dc5-8dfd-a98cc9f4af51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:18.782304 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4125c-8953-4dc5-8dfd-a98cc9f4af51 None admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:18.782573 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4125c-8953-4dc5-8dfd-a98cc9f4af51 None admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:18.782864 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4125c-8953-4dc5-8dfd-a98cc9f4af51 None admin] PATH_INFO: `/v3/services` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:18.783479 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4125c-8953-4dc5-8dfd-a98cc9f4af51 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:46:18.783779 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4125c-8953-4dc5-8dfd-a98cc9f4af51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:18.785170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4125c-8953-4dc5-8dfd-a98cc9f4af51 None admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:18.790613 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 661/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Sat Sep 10 23:46:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:46:18.796479 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989e4d3-7f30-4771-a84a-51a0219d8511 None admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:46:18.896804 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989e4d3-7f30-4771-a84a-51a0219d8511 None admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:46:18.935609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805533fd-899c-48e1-9a87-7d9b1154d656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:46:18.936198 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805533fd-899c-48e1-9a87-7d9b1154d656 None admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:46:18.936475 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805533fd-899c-48e1-9a87-7d9b1154d656 None admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:46:18.936744 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805533fd-899c-48e1-9a87-7d9b1154d656 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:46:18.937411 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805533fd-899c-48e1-9a87-7d9b1154d656 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:46:18.938835 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805533fd-899c-48e1-9a87-7d9b1154d656 None admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:46:19.054861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 645/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Sep 10 23:46:18 2022] POST /identity/v3/endpoints => generated 331 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 23:46:28.804720 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=30 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:46:29.050100 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:52:20.864659 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec71129-4138-4e0e-8318-a81d35288e07 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:20.865211 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec71129-4138-4e0e-8318-a81d35288e07 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:20.865211 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec71129-4138-4e0e-8318-a81d35288e07 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:20.871965 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf12ff-15f6-4f69-96c1-e41de42da39b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:20.878151 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf12ff-15f6-4f69-96c1-e41de42da39b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:20.878151 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf12ff-15f6-4f69-96c1-e41de42da39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:20.879324 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 662/1307] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:52:20 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:52:20.961337 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d9cf12ff-15f6-4f69-96c1-e41de42da39b None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:52:20.962960 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d9cf12ff-15f6-4f69-96c1-e41de42da39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:21.072737 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 646/1308] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:52:21.396793 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4730d-254d-410e-9d08-6aed9bab1b07 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:21.397154 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4730d-254d-410e-9d08-6aed9bab1b07 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:21.397340 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4730d-254d-410e-9d08-6aed9bab1b07 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:21.399316 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 663/1309] 10.222.0.17 () {54 vars in 965 bytes} [Sat Sep 10 23:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:52:21.412002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc35bff-a55f-41ad-a16e-8820600e2047 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:21.412322 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc35bff-a55f-41ad-a16e-8820600e2047 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:21.412711 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc35bff-a55f-41ad-a16e-8820600e2047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:21.527611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-dfc35bff-a55f-41ad-a16e-8820600e2047 None None] MFA Rules not processed for user `53e79d9fb2cd452d8d999a84b6425959`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:52:21.527611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-dfc35bff-a55f-41ad-a16e-8820600e2047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:21.549921 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:52:21.621979 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 647/1310] 10.222.0.17 () {58 vars in 1062 bytes} [Sat Sep 10 23:52:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:52:21.631607 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4730d-254d-410e-9d08-6aed9bab1b07 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:52:21.633360 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a5a4730d-254d-410e-9d08-6aed9bab1b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:21.760959 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4730d-254d-410e-9d08-6aed9bab1b07 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:52:21.845727 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47d59f-7301-4a74-8ace-f279d1e82580 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:52:21.846249 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47d59f-7301-4a74-8ace-f279d1e82580 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:21.846530 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47d59f-7301-4a74-8ace-f279d1e82580 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:21.846822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47d59f-7301-4a74-8ace-f279d1e82580 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:21.848425 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8c47d59f-7301-4a74-8ace-f279d1e82580 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:21.909132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:52:21.969394 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47d59f-7301-4a74-8ace-f279d1e82580 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:52:21.970857 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47d59f-7301-4a74-8ace-f279d1e82580 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:52:21.991274 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 664/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:52:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:52:24.559676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2339b-2bb5-43eb-a773-ba651fa68662 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:24.560203 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2339b-2bb5-43eb-a773-ba651fa68662 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:24.560203 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2339b-2bb5-43eb-a773-ba651fa68662 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:24.561483 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 648/1312] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:52:24.568097 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deea2bb-27fb-4fee-a3d2-e2cec8382157 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:24.568490 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deea2bb-27fb-4fee-a3d2-e2cec8382157 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:24.568638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deea2bb-27fb-4fee-a3d2-e2cec8382157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:24.649427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-6deea2bb-27fb-4fee-a3d2-e2cec8382157 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:52:24.650624 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6deea2bb-27fb-4fee-a3d2-e2cec8382157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:24.745746 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 665/1313] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:52:25.045122 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508cecb-c2a0-4536-bb3b-361fc81bc64c None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:25.045384 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508cecb-c2a0-4536-bb3b-361fc81bc64c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:25.045654 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508cecb-c2a0-4536-bb3b-361fc81bc64c None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:25.047195 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 649/1314] 10.222.0.17 () {54 vars in 965 bytes} [Sat Sep 10 23:52:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:52:25.053212 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b42a8-de2c-4485-87b2-60856955a72f None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:25.053515 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b42a8-de2c-4485-87b2-60856955a72f None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:25.053785 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b42a8-de2c-4485-87b2-60856955a72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:25.065685 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:52:25.127994 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-cc2b42a8-de2c-4485-87b2-60856955a72f None None] MFA Rules not processed for user `53e79d9fb2cd452d8d999a84b6425959`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:52:25.129181 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cc2b42a8-de2c-4485-87b2-60856955a72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:25.201936 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 666/1315] 10.222.0.17 () {58 vars in 1062 bytes} [Sat Sep 10 23:52:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:52:25.211248 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508cecb-c2a0-4536-bb3b-361fc81bc64c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:52:25.212767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f508cecb-c2a0-4536-bb3b-361fc81bc64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:25.322732 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508cecb-c2a0-4536-bb3b-361fc81bc64c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:52:25.393744 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:52:25.396011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d5a37-8671-487b-b666-c4c59e307095 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:52:25.396613 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d5a37-8671-487b-b666-c4c59e307095 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:25.396889 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d5a37-8671-487b-b666-c4c59e307095 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:25.397160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d5a37-8671-487b-b666-c4c59e307095 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:25.398760 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a40d5a37-8671-487b-b666-c4c59e307095 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:25.508991 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d5a37-8671-487b-b666-c4c59e307095 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:52:25.510421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d5a37-8671-487b-b666-c4c59e307095 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:52:25.527939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 650/1316] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:52:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 23:52:28.328545 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2bfc0-890f-4540-9c6e-0a071df3e90a None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:28.329057 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2bfc0-890f-4540-9c6e-0a071df3e90a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:28.329057 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2bfc0-890f-4540-9c6e-0a071df3e90a None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:28.330415 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 667/1317] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:52:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:52:28.335995 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac077554-1558-4b15-910d-ba1f5c1810e1 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:28.336746 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac077554-1558-4b15-910d-ba1f5c1810e1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:28.336902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac077554-1558-4b15-910d-ba1f5c1810e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:28.402040 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ac077554-1558-4b15-910d-ba1f5c1810e1 None None] MFA Rules not processed for user `067af36c9f9b401d9d35810237e9ba2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:52:28.403272 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ac077554-1558-4b15-910d-ba1f5c1810e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:28.456149 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:52:28.499485 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 651/1318] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:52:28 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:52:28.788441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2bfc0-890f-4540-9c6e-0a071df3e90a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:52:28.790044 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-edf2bfc0-890f-4540-9c6e-0a071df3e90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:28.893714 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2bfc0-890f-4540-9c6e-0a071df3e90a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:52:28.896570 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:52:28.971492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e88e1-a1a3-4795-b910-9a1967189c1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:52:28.972092 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e88e1-a1a3-4795-b910-9a1967189c1a service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:28.972366 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e88e1-a1a3-4795-b910-9a1967189c1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:28.972636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e88e1-a1a3-4795-b910-9a1967189c1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:28.974233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a73e88e1-a1a3-4795-b910-9a1967189c1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:29.081731 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e88e1-a1a3-4795-b910-9a1967189c1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:52:29.083118 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e88e1-a1a3-4795-b910-9a1967189c1a service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:52:29.101093 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 668/1319] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:52:28 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:52:32.199694 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bfa10-ebe1-425f-9e90-9815f2af46a5 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:32.200233 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bfa10-ebe1-425f-9e90-9815f2af46a5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:32.200233 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bfa10-ebe1-425f-9e90-9815f2af46a5 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:32.201579 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 652/1320] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:52:32.207419 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3978a6-5138-4601-83bd-8c40b3beb184 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:32.207847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3978a6-5138-4601-83bd-8c40b3beb184 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:32.207943 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3978a6-5138-4601-83bd-8c40b3beb184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:32.247283 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:52:32.271815 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-0c3978a6-5138-4601-83bd-8c40b3beb184 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:52:32.272999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0c3978a6-5138-4601-83bd-8c40b3beb184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:32.367622 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 669/1321] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:52:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:52:32.651442 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389bfa10-ebe1-425f-9e90-9815f2af46a5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:52:32.652986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-389bfa10-ebe1-425f-9e90-9815f2af46a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:32.753862 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389bfa10-ebe1-425f-9e90-9815f2af46a5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:52:32.826299 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19894c6-6e77-429e-b6e4-5d2a2a29c3db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:52:32.826921 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19894c6-6e77-429e-b6e4-5d2a2a29c3db service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:32.827197 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19894c6-6e77-429e-b6e4-5d2a2a29c3db service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:32.827463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19894c6-6e77-429e-b6e4-5d2a2a29c3db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:32.829248 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a19894c6-6e77-429e-b6e4-5d2a2a29c3db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:32.836066 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:52:32.937846 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19894c6-6e77-429e-b6e4-5d2a2a29c3db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:52:32.939315 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19894c6-6e77-429e-b6e4-5d2a2a29c3db service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:52:32.957192 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 653/1322] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:52:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:52:36.139974 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e19b7f-72a2-42c6-8f67-40b71d9beb2f None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:36.140517 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e19b7f-72a2-42c6-8f67-40b71d9beb2f None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:36.140517 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e19b7f-72a2-42c6-8f67-40b71d9beb2f None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:36.142455 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 670/1323] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:52:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:52:36.148090 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bafa8-dc6c-4d07-a50e-70dabd75546e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:36.148847 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bafa8-dc6c-4d07-a50e-70dabd75546e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:36.149136 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bafa8-dc6c-4d07-a50e-70dabd75546e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:36.215850 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-296bafa8-dc6c-4d07-a50e-70dabd75546e None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:52:36.217079 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-296bafa8-dc6c-4d07-a50e-70dabd75546e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:36.280595 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:52:36.326443 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 654/1324] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:52:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:52:36.606288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e19b7f-72a2-42c6-8f67-40b71d9beb2f None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:52:36.607844 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-24e19b7f-72a2-42c6-8f67-40b71d9beb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:36.708662 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e19b7f-72a2-42c6-8f67-40b71d9beb2f None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:52:36.721988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:52:36.783214 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd85ef4-15de-4cb4-a6b7-e8a36a6e9ec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:52:36.783813 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd85ef4-15de-4cb4-a6b7-e8a36a6e9ec3 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:36.784084 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd85ef4-15de-4cb4-a6b7-e8a36a6e9ec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:36.784349 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd85ef4-15de-4cb4-a6b7-e8a36a6e9ec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:36.785933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-afd85ef4-15de-4cb4-a6b7-e8a36a6e9ec3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:36.892884 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd85ef4-15de-4cb4-a6b7-e8a36a6e9ec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:52:36.894256 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd85ef4-15de-4cb4-a6b7-e8a36a6e9ec3 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:52:36.911842 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 671/1325] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:52:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 23:52:39.175996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e9eef-fc3d-48a1-8e24-0c431f14f449 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:39.176548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e9eef-fc3d-48a1-8e24-0c431f14f449 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:39.176548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e9eef-fc3d-48a1-8e24-0c431f14f449 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:39.178141 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 655/1326] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:52:39.183856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9721b6-6ce2-4d2a-a085-b3a9bb5b9db2 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:39.184405 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9721b6-6ce2-4d2a-a085-b3a9bb5b9db2 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:39.184405 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9721b6-6ce2-4d2a-a085-b3a9bb5b9db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:39.233284 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:52:39.252029 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-2f9721b6-6ce2-4d2a-a085-b3a9bb5b9db2 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:52:39.255604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2f9721b6-6ce2-4d2a-a085-b3a9bb5b9db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:39.355475 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 672/1327] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:52:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:52:39.655691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e9eef-fc3d-48a1-8e24-0c431f14f449 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:52:39.657839 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-437e9eef-fc3d-48a1-8e24-0c431f14f449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:39.761978 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e9eef-fc3d-48a1-8e24-0c431f14f449 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:52:39.836093 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705063cc-2c5f-496d-b7bf-e182d98d5856 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:52:39.836698 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705063cc-2c5f-496d-b7bf-e182d98d5856 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:39.836973 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705063cc-2c5f-496d-b7bf-e182d98d5856 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:39.837241 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705063cc-2c5f-496d-b7bf-e182d98d5856 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:39.838867 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-705063cc-2c5f-496d-b7bf-e182d98d5856 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:39.857401 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:52:39.965348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705063cc-2c5f-496d-b7bf-e182d98d5856 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:52:39.966830 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705063cc-2c5f-496d-b7bf-e182d98d5856 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:52:39.985363 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 656/1328] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:52:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:52:42.269165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c9337-1c7f-470a-899a-39ce4a03c493 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:42.269658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c9337-1c7f-470a-899a-39ce4a03c493 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:42.269658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c9337-1c7f-470a-899a-39ce4a03c493 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:42.271044 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 673/1329] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:52:42.276341 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15175fcb-84d0-402a-8a45-509e0cc56621 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:42.276764 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15175fcb-84d0-402a-8a45-509e0cc56621 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:42.276892 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15175fcb-84d0-402a-8a45-509e0cc56621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:42.344862 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-15175fcb-84d0-402a-8a45-509e0cc56621 None None] MFA Rules not processed for user `067af36c9f9b401d9d35810237e9ba2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:52:42.346017 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-15175fcb-84d0-402a-8a45-509e0cc56621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:42.411953 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:52:42.448930 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 657/1330] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:52:42 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 23:52:42.733625 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c9337-1c7f-470a-899a-39ce4a03c493 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:52:42.735320 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a64c9337-1c7f-470a-899a-39ce4a03c493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:42.851276 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c9337-1c7f-470a-899a-39ce4a03c493 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:52:42.874089 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:52:42.933384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c377d4-a8d0-448a-94e5-2aee2571d32e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:52:42.933979 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c377d4-a8d0-448a-94e5-2aee2571d32e service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:42.934250 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c377d4-a8d0-448a-94e5-2aee2571d32e service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:42.934513 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c377d4-a8d0-448a-94e5-2aee2571d32e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:42.936099 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-66c377d4-a8d0-448a-94e5-2aee2571d32e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:43.049008 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c377d4-a8d0-448a-94e5-2aee2571d32e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:52:43.050431 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c377d4-a8d0-448a-94e5-2aee2571d32e service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:52:43.069721 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 674/1331] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:52:42 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:52:45.642951 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e9d1a-8e93-4e0f-b4c9-c3e04eac5ed3 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:45.643461 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e9d1a-8e93-4e0f-b4c9-c3e04eac5ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:45.643461 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e9d1a-8e93-4e0f-b4c9-c3e04eac5ed3 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:45.644744 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 658/1332] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:52:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:52:45.649874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89779cd3-d661-4ed7-aa21-f2002acb22ad None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:45.650280 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89779cd3-d661-4ed7-aa21-f2002acb22ad None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:45.650418 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89779cd3-d661-4ed7-aa21-f2002acb22ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:45.712595 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:52:45.715391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-89779cd3-d661-4ed7-aa21-f2002acb22ad None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:52:45.716618 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-89779cd3-d661-4ed7-aa21-f2002acb22ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:45.811634 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 675/1333] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:52:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:52:46.101408 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e9d1a-8e93-4e0f-b4c9-c3e04eac5ed3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:52:46.102991 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7f6e9d1a-8e93-4e0f-b4c9-c3e04eac5ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:46.205471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e9d1a-8e93-4e0f-b4c9-c3e04eac5ed3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:52:46.276580 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fba4ce7-4fd3-49e5-ba0b-92aa23c07ee3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:52:46.277172 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba4ce7-4fd3-49e5-ba0b-92aa23c07ee3 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:46.277451 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba4ce7-4fd3-49e5-ba0b-92aa23c07ee3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:46.277720 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba4ce7-4fd3-49e5-ba0b-92aa23c07ee3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:46.279322 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8fba4ce7-4fd3-49e5-ba0b-92aa23c07ee3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:46.306830 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:52:46.386695 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fba4ce7-4fd3-49e5-ba0b-92aa23c07ee3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:52:46.388079 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fba4ce7-4fd3-49e5-ba0b-92aa23c07ee3 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:52:46.405935 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 659/1334] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:52:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:52:48.598595 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd9142-06f7-434d-8320-1bf9005211d2 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:48.599190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd9142-06f7-434d-8320-1bf9005211d2 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:48.599190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd9142-06f7-434d-8320-1bf9005211d2 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:48.600430 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 676/1335] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:52:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:52:48.605509 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21b245-8dfc-43aa-bc2c-6088b62ced2b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:48.605901 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21b245-8dfc-43aa-bc2c-6088b62ced2b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:48.606050 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21b245-8dfc-43aa-bc2c-6088b62ced2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:48.671033 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5c21b245-8dfc-43aa-bc2c-6088b62ced2b None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:52:48.672287 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5c21b245-8dfc-43aa-bc2c-6088b62ced2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:48.744581 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:52:48.766886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 660/1336] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:52:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:52:49.049655 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd9142-06f7-434d-8320-1bf9005211d2 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:52:49.051221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-dedd9142-06f7-434d-8320-1bf9005211d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:49.152851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd9142-06f7-434d-8320-1bf9005211d2 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:52:49.183232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:52:49.228213 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6e805-9335-4683-8de7-3759f51be733 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:52:49.228813 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6e805-9335-4683-8de7-3759f51be733 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:49.229086 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6e805-9335-4683-8de7-3759f51be733 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:49.229351 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6e805-9335-4683-8de7-3759f51be733 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:49.230938 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c5a6e805-9335-4683-8de7-3759f51be733 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:49.339840 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a6e805-9335-4683-8de7-3759f51be733 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:52:49.341282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a6e805-9335-4683-8de7-3759f51be733 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:52:49.358673 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 677/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:52:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:52:52.129360 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32444b7-31f6-409a-8571-342216465e5d None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:52.129931 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32444b7-31f6-409a-8571-342216465e5d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:52.129931 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32444b7-31f6-409a-8571-342216465e5d None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:52.131260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 661/1338] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:52:52.136354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c96ba6-bc4b-49cb-9870-08fa396be096 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:52.136808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c96ba6-bc4b-49cb-9870-08fa396be096 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:52.136911 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c96ba6-bc4b-49cb-9870-08fa396be096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:52.201974 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-64c96ba6-bc4b-49cb-9870-08fa396be096 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:52:52.203208 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-64c96ba6-bc4b-49cb-9870-08fa396be096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:52.211373 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:52:52.296580 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 678/1339] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:52:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:52:52.585294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32444b7-31f6-409a-8571-342216465e5d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:52:52.586854 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b32444b7-31f6-409a-8571-342216465e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:52.689572 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32444b7-31f6-409a-8571-342216465e5d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:52:52.763444 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5446b4-b5e9-4440-8739-aea54584cd0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:52:52.764092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5446b4-b5e9-4440-8739-aea54584cd0a service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:52.764366 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5446b4-b5e9-4440-8739-aea54584cd0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:52.764631 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5446b4-b5e9-4440-8739-aea54584cd0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:52.766283 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0d5446b4-b5e9-4440-8739-aea54584cd0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:52.803681 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:52:52.877592 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5446b4-b5e9-4440-8739-aea54584cd0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:52:52.879074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5446b4-b5e9-4440-8739-aea54584cd0a service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:52:52.897331 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 662/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:52:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:52:57.380174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d6771-27b0-4086-815a-3a4c6f62f727 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:57.380702 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d6771-27b0-4086-815a-3a4c6f62f727 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:57.380702 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d6771-27b0-4086-815a-3a4c6f62f727 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:57.382029 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 679/1341] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:52:57.387301 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849d2f4-a0db-418d-b53b-1db823615a26 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:57.387707 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849d2f4-a0db-418d-b53b-1db823615a26 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:57.387840 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849d2f4-a0db-418d-b53b-1db823615a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:57.460511 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-9849d2f4-a0db-418d-b53b-1db823615a26 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:52:57.461635 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9849d2f4-a0db-418d-b53b-1db823615a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:57.551377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:52:57.562268 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 663/1342] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:52:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:52:57.848637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d6771-27b0-4086-815a-3a4c6f62f727 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:52:57.850186 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cb7d6771-27b0-4086-815a-3a4c6f62f727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:57.956938 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d6771-27b0-4086-815a-3a4c6f62f727 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:52:57.995609 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:52:58.032095 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f7c5f-8038-443d-8c0e-33d1f1b50adb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:52:58.032696 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f7c5f-8038-443d-8c0e-33d1f1b50adb service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:52:58.032972 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f7c5f-8038-443d-8c0e-33d1f1b50adb service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:52:58.033231 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f7c5f-8038-443d-8c0e-33d1f1b50adb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:52:58.034846 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-fb9f7c5f-8038-443d-8c0e-33d1f1b50adb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:52:58.145298 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f7c5f-8038-443d-8c0e-33d1f1b50adb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:52:58.146670 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f7c5f-8038-443d-8c0e-33d1f1b50adb service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:52:58.164896 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 680/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:52:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:53:01.285649 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05c2fd-72c4-4291-aa70-26ab6c07d0a8 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:01.286174 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05c2fd-72c4-4291-aa70-26ab6c07d0a8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:01.286174 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05c2fd-72c4-4291-aa70-26ab6c07d0a8 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:01.287609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 664/1344] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:53:01.292846 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1c515-9166-4ae2-8bac-98c357a9aa13 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:01.293295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1c515-9166-4ae2-8bac-98c357a9aa13 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:01.293391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1c515-9166-4ae2-8bac-98c357a9aa13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:01.366953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-f9b1c515-9166-4ae2-8bac-98c357a9aa13 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:53:01.368153 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f9b1c515-9166-4ae2-8bac-98c357a9aa13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:01.386108 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:01.467714 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 681/1345] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:53:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:53:01.753797 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05c2fd-72c4-4291-aa70-26ab6c07d0a8 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:53:01.755479 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6c05c2fd-72c4-4291-aa70-26ab6c07d0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:01.863130 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05c2fd-72c4-4291-aa70-26ab6c07d0a8 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:53:01.938017 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26922b38-fb48-41cd-8df8-45166da5320f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:53:01.938738 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26922b38-fb48-41cd-8df8-45166da5320f service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:01.941628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26922b38-fb48-41cd-8df8-45166da5320f service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:01.941628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26922b38-fb48-41cd-8df8-45166da5320f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:01.941628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-26922b38-fb48-41cd-8df8-45166da5320f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:01.986976 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:02.067981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26922b38-fb48-41cd-8df8-45166da5320f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:53:02.069500 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26922b38-fb48-41cd-8df8-45166da5320f service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:53:02.090368 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 665/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:53:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:53:06.722596 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec07f7f-bd97-4d41-8f75-0dc9dc1161cb None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:06.723159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec07f7f-bd97-4d41-8f75-0dc9dc1161cb None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:06.723253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec07f7f-bd97-4d41-8f75-0dc9dc1161cb None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:06.724531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 682/1347] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:53:06.730023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee22d33-4d67-430d-a6ae-ed52899f8747 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:06.730639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee22d33-4d67-430d-a6ae-ed52899f8747 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:06.730639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee22d33-4d67-430d-a6ae-ed52899f8747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:06.807435 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-eee22d33-4d67-430d-a6ae-ed52899f8747 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:53:06.808695 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-eee22d33-4d67-430d-a6ae-ed52899f8747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:06.908429 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:06.910585 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 666/1348] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:53:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:53:07.191901 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec07f7f-bd97-4d41-8f75-0dc9dc1161cb None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:53:07.193426 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-fec07f7f-bd97-4d41-8f75-0dc9dc1161cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:07.299408 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec07f7f-bd97-4d41-8f75-0dc9dc1161cb None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:53:07.347730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:07.372692 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ed0a8-14b5-47f0-80a4-5ddbf970518a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:53:07.373285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ed0a8-14b5-47f0-80a4-5ddbf970518a service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:07.373555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ed0a8-14b5-47f0-80a4-5ddbf970518a service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:07.373819 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ed0a8-14b5-47f0-80a4-5ddbf970518a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:07.375580 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-813ed0a8-14b5-47f0-80a4-5ddbf970518a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:07.485574 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ed0a8-14b5-47f0-80a4-5ddbf970518a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:53:07.487028 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ed0a8-14b5-47f0-80a4-5ddbf970518a service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:53:07.506165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 683/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:53:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:53:12.621740 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34649921-217e-46ae-a977-6cfbee87c824 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:12.622284 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34649921-217e-46ae-a977-6cfbee87c824 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:12.622284 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34649921-217e-46ae-a977-6cfbee87c824 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:12.623890 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 667/1350] 10.222.0.17 () {54 vars in 921 bytes} [Sat Sep 10 23:53:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:53:12.633305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773e307-369e-4eae-a7d2-e43c145149fb None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:12.634229 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773e307-369e-4eae-a7d2-e43c145149fb None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:12.635546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773e307-369e-4eae-a7d2-e43c145149fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:12.753077 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b773e307-369e-4eae-a7d2-e43c145149fb None None] MFA Rules not processed for user `5cb1d85a338e40f18e61887a1becb906`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:53:12.754279 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b773e307-369e-4eae-a7d2-e43c145149fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:12.773538 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:12.875306 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 684/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Sep 10 23:53:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 23:53:17.023015 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822db988-017f-4d37-ad85-d403bbba9b7d None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:17.023529 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822db988-017f-4d37-ad85-d403bbba9b7d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:17.023529 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822db988-017f-4d37-ad85-d403bbba9b7d None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:17.024915 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 668/1352] 10.222.0.17 () {54 vars in 921 bytes} [Sat Sep 10 23:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:53:17.030102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd078f7-2862-4e60-a104-536fb4f05147 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:17.031014 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd078f7-2862-4e60-a104-536fb4f05147 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:17.031529 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd078f7-2862-4e60-a104-536fb4f05147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:17.111953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-7bd078f7-2862-4e60-a104-536fb4f05147 None None] MFA Rules not processed for user `5cb1d85a338e40f18e61887a1becb906`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:53:17.113602 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-7bd078f7-2862-4e60-a104-536fb4f05147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:17.192865 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:17.229529 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 685/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Sep 10 23:53:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:53:18.745662 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005f3e4-f90a-4061-a5c2-bb61b18e848b None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:18.746321 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005f3e4-f90a-4061-a5c2-bb61b18e848b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:18.746321 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005f3e4-f90a-4061-a5c2-bb61b18e848b None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:18.747630 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 669/1354] 10.222.0.17 () {54 vars in 921 bytes} [Sat Sep 10 23:53:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:53:18.753374 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622af89-0d33-4382-9ad7-8ee80aca46f9 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:18.753810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622af89-0d33-4382-9ad7-8ee80aca46f9 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:18.754065 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622af89-0d33-4382-9ad7-8ee80aca46f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:18.832092 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-0622af89-0d33-4382-9ad7-8ee80aca46f9 None None] MFA Rules not processed for user `5cb1d85a338e40f18e61887a1becb906`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:53:18.833296 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0622af89-0d33-4382-9ad7-8ee80aca46f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:18.944791 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 686/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Sat Sep 10 23:53:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:53:22.474873 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cee29c-46ba-4d08-b6bc-3580a7b718d3 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:22.475723 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cee29c-46ba-4d08-b6bc-3580a7b718d3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:22.476171 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cee29c-46ba-4d08-b6bc-3580a7b718d3 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:22.478493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 670/1356] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:53:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:53:22.483235 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c67fa9-e471-4abb-82fa-b8361c882472 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:22.484199 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c67fa9-e471-4abb-82fa-b8361c882472 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:22.484691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c67fa9-e471-4abb-82fa-b8361c882472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:22.530169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:22.568127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-10c67fa9-e471-4abb-82fa-b8361c882472 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:53:22.570034 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-10c67fa9-e471-4abb-82fa-b8361c882472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:22.682786 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 687/1357] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 23:53:22.982214 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c07a2a-17dc-4d2b-86ed-7430e352ec08 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:22.982495 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c07a2a-17dc-4d2b-86ed-7430e352ec08 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:22.982883 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c07a2a-17dc-4d2b-86ed-7430e352ec08 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:22.985139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 671/1358] 10.222.0.17 () {54 vars in 962 bytes} [Sat Sep 10 23:53:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:53:22.992327 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a5c8f-ea82-4c60-a057-40bfb4fb4122 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:22.992843 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a5c8f-ea82-4c60-a057-40bfb4fb4122 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:22.993278 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a5c8f-ea82-4c60-a057-40bfb4fb4122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:23.075451 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-672a5c8f-ea82-4c60-a057-40bfb4fb4122 None None] MFA Rules not processed for user `636bfafb693044d283530063047fc0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:53:23.076941 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-672a5c8f-ea82-4c60-a057-40bfb4fb4122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:23.129886 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:23.199283 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 688/1359] 10.222.0.17 () {58 vars in 1059 bytes} [Sat Sep 10 23:53:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:53:23.208342 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c07a2a-17dc-4d2b-86ed-7430e352ec08 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:53:23.209949 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-64c07a2a-17dc-4d2b-86ed-7430e352ec08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:23.382045 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c07a2a-17dc-4d2b-86ed-7430e352ec08 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:53:23.460109 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:23.501123 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41109318-5d8d-431c-989e-b64b51dab62d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:53:23.501783 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41109318-5d8d-431c-989e-b64b51dab62d service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:23.502035 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41109318-5d8d-431c-989e-b64b51dab62d service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:23.502462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41109318-5d8d-431c-989e-b64b51dab62d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:23.504177 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-41109318-5d8d-431c-989e-b64b51dab62d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:23.599810 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41109318-5d8d-431c-989e-b64b51dab62d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:53:23.601450 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41109318-5d8d-431c-989e-b64b51dab62d service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:53:23.623396 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 672/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:53:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:53:26.040324 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064bb78-ab8d-4e70-a065-a3c5e967c352 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:26.040875 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064bb78-ab8d-4e70-a065-a3c5e967c352 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:26.040875 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064bb78-ab8d-4e70-a065-a3c5e967c352 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:26.042381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 689/1361] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:53:26.048073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfd581-d686-40ac-9860-c312c7d915a4 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:26.048510 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfd581-d686-40ac-9860-c312c7d915a4 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:26.048604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfd581-d686-40ac-9860-c312c7d915a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:26.121443 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-49bfd581-d686-40ac-9860-c312c7d915a4 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:53:26.122792 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-49bfd581-d686-40ac-9860-c312c7d915a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:26.161098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:26.223262 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 673/1362] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:53:26.514586 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064bb78-ab8d-4e70-a065-a3c5e967c352 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:53:26.517590 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f064bb78-ab8d-4e70-a065-a3c5e967c352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:26.679987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064bb78-ab8d-4e70-a065-a3c5e967c352 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:53:26.691130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:26.820117 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3399638-ad27-4a80-bc5a-262f14db73c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:53:26.820946 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3399638-ad27-4a80-bc5a-262f14db73c1 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:26.821223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3399638-ad27-4a80-bc5a-262f14db73c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:26.821489 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3399638-ad27-4a80-bc5a-262f14db73c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:26.823125 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a3399638-ad27-4a80-bc5a-262f14db73c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:26.924636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3399638-ad27-4a80-bc5a-262f14db73c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:53:26.926252 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3399638-ad27-4a80-bc5a-262f14db73c1 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:53:26.945579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 690/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:53:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:53:29.561292 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207e971-60bc-421e-8e98-a34ee47cfeb6 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:29.562437 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207e971-60bc-421e-8e98-a34ee47cfeb6 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:29.562437 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207e971-60bc-421e-8e98-a34ee47cfeb6 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:29.563957 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 674/1364] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:53:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:53:29.569237 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1fa055-83e3-48cf-b617-275b99d48328 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:29.569631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1fa055-83e3-48cf-b617-275b99d48328 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:29.569814 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1fa055-83e3-48cf-b617-275b99d48328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:29.585688 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:29.637481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-3f1fa055-83e3-48cf-b617-275b99d48328 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:53:29.638766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3f1fa055-83e3-48cf-b617-275b99d48328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:29.742110 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 691/1365] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:53:30.035692 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59682c-b0e8-4abf-bbbe-e27ed3af1d25 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:30.036240 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59682c-b0e8-4abf-bbbe-e27ed3af1d25 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:30.036676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59682c-b0e8-4abf-bbbe-e27ed3af1d25 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:30.038173 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 675/1366] 10.222.0.17 () {54 vars in 962 bytes} [Sat Sep 10 23:53:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:53:30.043466 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991da2a-94ef-47f1-987d-0336f81ec9d3 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:30.043750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991da2a-94ef-47f1-987d-0336f81ec9d3 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:30.044023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991da2a-94ef-47f1-987d-0336f81ec9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:30.133200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-5991da2a-94ef-47f1-987d-0336f81ec9d3 None None] MFA Rules not processed for user `636bfafb693044d283530063047fc0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:53:30.134519 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5991da2a-94ef-47f1-987d-0336f81ec9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:30.171620 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:30.256322 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 692/1367] 10.222.0.17 () {58 vars in 1059 bytes} [Sat Sep 10 23:53:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:53:30.264242 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59682c-b0e8-4abf-bbbe-e27ed3af1d25 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:53:30.266422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2b59682c-b0e8-4abf-bbbe-e27ed3af1d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:30.433067 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59682c-b0e8-4abf-bbbe-e27ed3af1d25 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:53:30.458452 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:30.558370 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a5dbb-1afc-44c8-a2be-cf592e48c040 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:53:30.559033 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a5dbb-1afc-44c8-a2be-cf592e48c040 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:30.559359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a5dbb-1afc-44c8-a2be-cf592e48c040 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:30.559710 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a5dbb-1afc-44c8-a2be-cf592e48c040 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:30.561345 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6d9a5dbb-1afc-44c8-a2be-cf592e48c040 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:30.659361 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a5dbb-1afc-44c8-a2be-cf592e48c040 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:53:30.661071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a5dbb-1afc-44c8-a2be-cf592e48c040 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:53:30.679900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 676/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:53:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:53:30.870309 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804aaaf-c9ce-4aa7-8605-d592c2b77c17 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:30.870881 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804aaaf-c9ce-4aa7-8605-d592c2b77c17 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:30.870881 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804aaaf-c9ce-4aa7-8605-d592c2b77c17 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:30.872241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 693/1369] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:53:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:53:30.877763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154cc5c-7033-44a2-99a6-4e071914771d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:30.878047 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154cc5c-7033-44a2-99a6-4e071914771d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:30.878318 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154cc5c-7033-44a2-99a6-4e071914771d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:30.912934 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:30.955331 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-8154cc5c-7033-44a2-99a6-4e071914771d None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:53:30.956737 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8154cc5c-7033-44a2-99a6-4e071914771d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:31.074430 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 677/1370] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:53:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:53:31.409752 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209ae2d-df40-499b-bae1-e9c727f1c9e6 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:31.410031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209ae2d-df40-499b-bae1-e9c727f1c9e6 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:31.410302 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209ae2d-df40-499b-bae1-e9c727f1c9e6 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:31.411802 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 694/1371] 10.222.0.17 () {54 vars in 964 bytes} [Sat Sep 10 23:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:53:31.416489 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbab75-2aca-4a19-9911-837f95e303f1 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:31.416935 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbab75-2aca-4a19-9911-837f95e303f1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:31.417331 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbab75-2aca-4a19-9911-837f95e303f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:31.504577 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-aefbab75-2aca-4a19-9911-837f95e303f1 None None] MFA Rules not processed for user `c6fccf47e1ec4324983c5218ec3c96a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:53:31.505924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-aefbab75-2aca-4a19-9911-837f95e303f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:31.563885 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:31.593564 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 678/1372] 10.222.0.17 () {58 vars in 1061 bytes} [Sat Sep 10 23:53:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 23:53:31.601567 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209ae2d-df40-499b-bae1-e9c727f1c9e6 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:53:31.603279 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2209ae2d-df40-499b-bae1-e9c727f1c9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:31.716064 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209ae2d-df40-499b-bae1-e9c727f1c9e6 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:53:31.763236 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:31.804575 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef94b9-38ac-4153-aa78-5e6deb67d8ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:53:31.805207 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef94b9-38ac-4153-aa78-5e6deb67d8ec service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:31.805486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef94b9-38ac-4153-aa78-5e6deb67d8ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:31.805758 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef94b9-38ac-4153-aa78-5e6deb67d8ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:31.807532 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-27ef94b9-38ac-4153-aa78-5e6deb67d8ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:31.923482 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef94b9-38ac-4153-aa78-5e6deb67d8ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:53:31.925127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef94b9-38ac-4153-aa78-5e6deb67d8ec service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:53:31.960079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 695/1373] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:53:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 23:53:33.087432 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7444dcf-90e0-4bcf-aa95-d2210c48b837 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:33.087960 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7444dcf-90e0-4bcf-aa95-d2210c48b837 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:33.088085 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7444dcf-90e0-4bcf-aa95-d2210c48b837 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:33.089365 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 679/1374] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:53:33.095376 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4fe03-651a-4464-a2e4-362662047500 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:33.095832 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4fe03-651a-4464-a2e4-362662047500 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:33.096035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4fe03-651a-4464-a2e4-362662047500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:33.162647 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-04f4fe03-651a-4464-a2e4-362662047500 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:53:33.163871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-04f4fe03-651a-4464-a2e4-362662047500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:33.184834 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:33.261709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 696/1375] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 23:53:33.555533 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7444dcf-90e0-4bcf-aa95-d2210c48b837 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:53:33.557035 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d7444dcf-90e0-4bcf-aa95-d2210c48b837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:33.724982 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7444dcf-90e0-4bcf-aa95-d2210c48b837 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:53:33.781618 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:33.847981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8940e1-3126-4670-91b0-b34c73399014 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:53:33.848497 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8940e1-3126-4670-91b0-b34c73399014 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:33.848771 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8940e1-3126-4670-91b0-b34c73399014 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:33.849044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8940e1-3126-4670-91b0-b34c73399014 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:33.850606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fa8940e1-3126-4670-91b0-b34c73399014 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:33.944979 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8940e1-3126-4670-91b0-b34c73399014 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:53:33.946583 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8940e1-3126-4670-91b0-b34c73399014 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:53:33.964841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 680/1376] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:53:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:53:36.329423 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fee67-4402-4068-b12d-80ef05d30eba None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:36.329922 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fee67-4402-4068-b12d-80ef05d30eba None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:36.330002 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fee67-4402-4068-b12d-80ef05d30eba None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:36.331438 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 697/1377] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:53:36.336811 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30e36a-f8cb-4d80-97dd-cfe50eba24a9 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:36.337257 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30e36a-f8cb-4d80-97dd-cfe50eba24a9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:36.337546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30e36a-f8cb-4d80-97dd-cfe50eba24a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:36.408652 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1c30e36a-f8cb-4d80-97dd-cfe50eba24a9 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:53:36.409984 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1c30e36a-f8cb-4d80-97dd-cfe50eba24a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:36.418632 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:36.519388 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 681/1378] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:53:36.811491 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915fee67-4402-4068-b12d-80ef05d30eba None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:53:36.813113 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-915fee67-4402-4068-b12d-80ef05d30eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:36.964413 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915fee67-4402-4068-b12d-80ef05d30eba None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:53:36.969218 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:37.081095 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf54e9b-5185-405a-ac0f-8af4ac88c90c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:53:37.081618 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf54e9b-5185-405a-ac0f-8af4ac88c90c service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:37.081894 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf54e9b-5185-405a-ac0f-8af4ac88c90c service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:37.082168 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf54e9b-5185-405a-ac0f-8af4ac88c90c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:37.083768 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-faf54e9b-5185-405a-ac0f-8af4ac88c90c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:37.176817 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf54e9b-5185-405a-ac0f-8af4ac88c90c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:53:37.178415 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf54e9b-5185-405a-ac0f-8af4ac88c90c service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:53:37.197861 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 698/1379] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:53:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:53:39.540895 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192352f8-3c8f-4c7f-9846-ce37541e10d1 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:39.540895 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192352f8-3c8f-4c7f-9846-ce37541e10d1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:39.540895 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192352f8-3c8f-4c7f-9846-ce37541e10d1 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:39.540895 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 682/1380] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:53:39.543340 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10561511-8641-4830-b63a-61d59634ccdd None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:39.543747 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10561511-8641-4830-b63a-61d59634ccdd None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:39.543885 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10561511-8641-4830-b63a-61d59634ccdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:39.556541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:39.618648 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-10561511-8641-4830-b63a-61d59634ccdd None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:53:39.619902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-10561511-8641-4830-b63a-61d59634ccdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:39.732031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 699/1381] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:53:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:53:40.053159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192352f8-3c8f-4c7f-9846-ce37541e10d1 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:53:40.054818 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-192352f8-3c8f-4c7f-9846-ce37541e10d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:40.206008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:40.216205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192352f8-3c8f-4c7f-9846-ce37541e10d1 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:53:40.354680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c26d8-93fa-4364-bf25-db75b8c435b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:53:40.355490 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c26d8-93fa-4364-bf25-db75b8c435b6 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:40.355822 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c26d8-93fa-4364-bf25-db75b8c435b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:40.356155 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c26d8-93fa-4364-bf25-db75b8c435b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:40.358148 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-486c26d8-93fa-4364-bf25-db75b8c435b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:40.466467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c26d8-93fa-4364-bf25-db75b8c435b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:53:40.468994 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c26d8-93fa-4364-bf25-db75b8c435b6 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:53:40.479800 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:40.489373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 683/1382] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:53:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:53:43.039751 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63835358-952e-46ad-8e54-132e62df82b1 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:43.040300 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63835358-952e-46ad-8e54-132e62df82b1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:43.040300 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63835358-952e-46ad-8e54-132e62df82b1 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:43.041624 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 700/1383] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:53:43.047940 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b981c-4c2e-4af1-ad66-35ded0aa6874 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:43.048466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b981c-4c2e-4af1-ad66-35ded0aa6874 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:43.048466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b981c-4c2e-4af1-ad66-35ded0aa6874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:43.118410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-366b981c-4c2e-4af1-ad66-35ded0aa6874 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:53:43.119668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-366b981c-4c2e-4af1-ad66-35ded0aa6874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:43.222441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 684/1384] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:53:43.528906 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63835358-952e-46ad-8e54-132e62df82b1 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:53:43.531296 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-63835358-952e-46ad-8e54-132e62df82b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:43.633137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:43.698741 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63835358-952e-46ad-8e54-132e62df82b1 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:53:43.818381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ddeadf-2875-48e2-a405-1eb4f31418af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:53:43.819033 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddeadf-2875-48e2-a405-1eb4f31418af service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:43.819364 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddeadf-2875-48e2-a405-1eb4f31418af service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:43.819631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddeadf-2875-48e2-a405-1eb4f31418af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:43.821257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-98ddeadf-2875-48e2-a405-1eb4f31418af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:43.880071 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:43.925274 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ddeadf-2875-48e2-a405-1eb4f31418af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:53:43.927147 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ddeadf-2875-48e2-a405-1eb4f31418af service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:53:43.949424 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 701/1385] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:53:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:53:46.576014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b038b47-8721-4b08-b1f6-0b91b2220f2b None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:46.576523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b038b47-8721-4b08-b1f6-0b91b2220f2b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:46.576523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b038b47-8721-4b08-b1f6-0b91b2220f2b None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:46.578392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 685/1386] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:53:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:53:46.583607 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efb6d6-b561-4a42-80c3-972946f5fa37 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:46.584042 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efb6d6-b561-4a42-80c3-972946f5fa37 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:46.584156 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efb6d6-b561-4a42-80c3-972946f5fa37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:46.650235 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-c1efb6d6-b561-4a42-80c3-972946f5fa37 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:53:46.651455 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c1efb6d6-b561-4a42-80c3-972946f5fa37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:46.749759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 702/1387] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:53:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:53:47.038183 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b038b47-8721-4b08-b1f6-0b91b2220f2b None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:53:47.039729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8b038b47-8721-4b08-b1f6-0b91b2220f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:47.118219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:47.190934 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b038b47-8721-4b08-b1f6-0b91b2220f2b None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:53:47.428349 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426efa05-2209-463a-aea3-ad9d5b428932 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:53:47.428951 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426efa05-2209-463a-aea3-ad9d5b428932 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:47.429224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426efa05-2209-463a-aea3-ad9d5b428932 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:47.429490 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426efa05-2209-463a-aea3-ad9d5b428932 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:47.431081 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-426efa05-2209-463a-aea3-ad9d5b428932 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:47.475851 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:47.532947 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426efa05-2209-463a-aea3-ad9d5b428932 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:53:47.534600 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426efa05-2209-463a-aea3-ad9d5b428932 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:53:47.553862 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 686/1388] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:53:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:53:47.586090 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a022d12-58ad-4ee1-af5a-595389e23396 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:47.586595 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a022d12-58ad-4ee1-af5a-595389e23396 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:47.586749 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a022d12-58ad-4ee1-af5a-595389e23396 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:47.588019 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 703/1389] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:53:47.593242 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867839dd-0075-498f-9462-0837c5639b67 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:47.593634 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867839dd-0075-498f-9462-0837c5639b67 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:47.593785 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867839dd-0075-498f-9462-0837c5639b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:47.663416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-867839dd-0075-498f-9462-0837c5639b67 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:53:47.664875 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-867839dd-0075-498f-9462-0837c5639b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:47.766240 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 687/1390] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:53:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:53:48.068537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7aebb-80be-47cc-9820-c16c5611b3b1 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:48.068813 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7aebb-80be-47cc-9820-c16c5611b3b1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:48.069103 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7aebb-80be-47cc-9820-c16c5611b3b1 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:48.070414 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 704/1391] 10.222.0.17 () {54 vars in 966 bytes} [Sat Sep 10 23:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:53:48.075934 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba7edd-94f6-44d6-8523-b658a0188134 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:48.076239 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba7edd-94f6-44d6-8523-b658a0188134 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:48.076512 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba7edd-94f6-44d6-8523-b658a0188134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:48.094085 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:48.156422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-86ba7edd-94f6-44d6-8523-b658a0188134 None None] MFA Rules not processed for user `2cb18199dc8b42589184042556e98983`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:53:48.157629 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-86ba7edd-94f6-44d6-8523-b658a0188134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:48.237164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 688/1392] 10.222.0.17 () {58 vars in 1063 bytes} [Sat Sep 10 23:53:48 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:53:48.245986 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7aebb-80be-47cc-9820-c16c5611b3b1 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:53:48.247634 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4ee7aebb-80be-47cc-9820-c16c5611b3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:48.278494 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:48.352960 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7aebb-80be-47cc-9820-c16c5611b3b1 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:53:48.427922 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573896c7-0c5f-4bdc-b145-73dba9e1235d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb18199dc8b42589184042556e98983', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:53:48.428537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573896c7-0c5f-4bdc-b145-73dba9e1235d service glance] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:48.429169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573896c7-0c5f-4bdc-b145-73dba9e1235d service glance] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:48.429169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573896c7-0c5f-4bdc-b145-73dba9e1235d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:48.431501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-573896c7-0c5f-4bdc-b145-73dba9e1235d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:48.519107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:48.552770 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573896c7-0c5f-4bdc-b145-73dba9e1235d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:53:48.554247 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573896c7-0c5f-4bdc-b145-73dba9e1235d service glance] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:53:48.574850 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 705/1393] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:53:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:53:49.446433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b26a3f-5b43-48fc-a1c9-b71031820b7d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:49.448349 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b26a3f-5b43-48fc-a1c9-b71031820b7d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:49.448349 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b26a3f-5b43-48fc-a1c9-b71031820b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:49.522552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-04b26a3f-5b43-48fc-a1c9-b71031820b7d None None] MFA Rules not processed for user `b0a879085a194b0687e1cdcbafd48033`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:53:49.523776 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-04b26a3f-5b43-48fc-a1c9-b71031820b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:49.614058 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:49.616750 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 689/1394] 10.222.0.17 () {58 vars in 1009 bytes} [Sat Sep 10 23:53:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:53:49.661386 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00225ca-295f-4563-b7e5-f9a989aec1b6 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:49.661910 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00225ca-295f-4563-b7e5-f9a989aec1b6 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:49.661993 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00225ca-295f-4563-b7e5-f9a989aec1b6 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:49.663363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 706/1395] 10.222.0.17 () {54 vars in 943 bytes} [Sat Sep 10 23:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:53:49.671883 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf54d5b-da22-440e-9830-ba26c7dc5f44 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:49.672340 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf54d5b-da22-440e-9830-ba26c7dc5f44 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:49.672593 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf54d5b-da22-440e-9830-ba26c7dc5f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:49.766233 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-8cf54d5b-da22-440e-9830-ba26c7dc5f44 None None] MFA Rules not processed for user `564e65585e9543e286b6041af870cf41`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:53:49.767488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8cf54d5b-da22-440e-9830-ba26c7dc5f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:49.852929 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14399d6-2d90-483a-af6e-aee9cc4d3eed None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:49.853202 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14399d6-2d90-483a-af6e-aee9cc4d3eed None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:49.853471 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14399d6-2d90-483a-af6e-aee9cc4d3eed None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:49.854825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 707/1396] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:53:49.860522 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2638d6a-f707-4a6e-a1dc-aff2e8bd42b9 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:49.860874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2638d6a-f707-4a6e-a1dc-aff2e8bd42b9 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:49.861241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2638d6a-f707-4a6e-a1dc-aff2e8bd42b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:49.880982 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 690/1397] 10.222.0.17 () {58 vars in 1040 bytes} [Sat Sep 10 23:53:49 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:53:49.890300 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf54d5b-da22-440e-9830-ba26c7dc5f44 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:53:49.899611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf54d5b-da22-440e-9830-ba26c7dc5f44 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:53:49.926410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:49.929374 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-c2638d6a-f707-4a6e-a1dc-aff2e8bd42b9 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:53:49.930598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c2638d6a-f707-4a6e-a1dc-aff2e8bd42b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:50.013941 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84c388-9dfc-4ffb-9627-b44e305ed92e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '564e65585e9543e286b6041af870cf41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:53:50.014607 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84c388-9dfc-4ffb-9627-b44e305ed92e service swift] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:50.014914 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84c388-9dfc-4ffb-9627-b44e305ed92e service swift] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:50.015186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84c388-9dfc-4ffb-9627-b44e305ed92e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:50.016813 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9a84c388-9dfc-4ffb-9627-b44e305ed92e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:50.037457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 708/1398] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 23:53:50.101556 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84c388-9dfc-4ffb-9627-b44e305ed92e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:53:50.103178 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84c388-9dfc-4ffb-9627-b44e305ed92e service swift] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:53:50.111495 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 691/1399] 10.222.0.17 () {60 vars in 1384 bytes} [Sat Sep 10 23:53:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 10 23:53:50.328981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2638d6a-f707-4a6e-a1dc-aff2e8bd42b9 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:53:50.330195 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c2638d6a-f707-4a6e-a1dc-aff2e8bd42b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:50.367816 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:50.444259 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2638d6a-f707-4a6e-a1dc-aff2e8bd42b9 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:53:50.584448 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043c952-949e-46d7-bb37-d7b9bc06aeec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:53:50.585064 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043c952-949e-46d7-bb37-d7b9bc06aeec service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:50.585407 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043c952-949e-46d7-bb37-d7b9bc06aeec service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:50.585741 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043c952-949e-46d7-bb37-d7b9bc06aeec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:50.587578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e043c952-949e-46d7-bb37-d7b9bc06aeec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:50.631345 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:50.694755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043c952-949e-46d7-bb37-d7b9bc06aeec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:53:50.696369 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043c952-949e-46d7-bb37-d7b9bc06aeec service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:53:50.716942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 709/1400] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:53:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:53:53.128150 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0c713-67bb-4fe4-aa88-11f0ab6af37d None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:53.128767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0c713-67bb-4fe4-aa88-11f0ab6af37d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:53.128767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0c713-67bb-4fe4-aa88-11f0ab6af37d None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:53.129968 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 692/1401] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:53:53.135330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe96401-e084-48dc-bf03-9c3393f932e0 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:53.135751 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe96401-e084-48dc-bf03-9c3393f932e0 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:53.135911 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe96401-e084-48dc-bf03-9c3393f932e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:53.216011 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-6fe96401-e084-48dc-bf03-9c3393f932e0 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:53:53.217243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6fe96401-e084-48dc-bf03-9c3393f932e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:53.347649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 710/1402] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:53:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:53:53.647508 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0c713-67bb-4fe4-aa88-11f0ab6af37d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:53:53.649000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-14a0c713-67bb-4fe4-aa88-11f0ab6af37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:53.716300 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:53.804452 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0c713-67bb-4fe4-aa88-11f0ab6af37d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:53:53.935244 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce08075d-c30f-4609-a6c5-8f2005efdbd1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:53:53.935850 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08075d-c30f-4609-a6c5-8f2005efdbd1 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:53.936185 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08075d-c30f-4609-a6c5-8f2005efdbd1 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:53.936459 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08075d-c30f-4609-a6c5-8f2005efdbd1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:53.938172 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ce08075d-c30f-4609-a6c5-8f2005efdbd1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:53.958608 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:54.039846 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce08075d-c30f-4609-a6c5-8f2005efdbd1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:53:54.041598 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce08075d-c30f-4609-a6c5-8f2005efdbd1 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:53:54.060697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 693/1403] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:53:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:53:56.446452 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d833fe-52a2-488b-b861-3d19473d0330 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:56.447506 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d833fe-52a2-488b-b861-3d19473d0330 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:56.448011 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d833fe-52a2-488b-b861-3d19473d0330 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:56.449865 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 711/1404] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:53:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:53:56.455283 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4012c-fda3-4d28-ae46-8dafdf2898e9 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:56.461233 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4012c-fda3-4d28-ae46-8dafdf2898e9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:56.461233 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4012c-fda3-4d28-ae46-8dafdf2898e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:56.529941 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-03c4012c-fda3-4d28-ae46-8dafdf2898e9 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:53:56.531433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-03c4012c-fda3-4d28-ae46-8dafdf2898e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:56.653564 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:56.660085 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 694/1405] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:53:56.957210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d833fe-52a2-488b-b861-3d19473d0330 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:53:56.958879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-34d833fe-52a2-488b-b861-3d19473d0330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:56.969678 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:57.123363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d833fe-52a2-488b-b861-3d19473d0330 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:53:57.210682 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:53:57.251891 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe002ffe-714c-4885-9805-352a82ba9988 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:53:57.252501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe002ffe-714c-4885-9805-352a82ba9988 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:57.252776 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe002ffe-714c-4885-9805-352a82ba9988 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:57.253039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe002ffe-714c-4885-9805-352a82ba9988 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:57.254683 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-fe002ffe-714c-4885-9805-352a82ba9988 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:57.359359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe002ffe-714c-4885-9805-352a82ba9988 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:53:57.361091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe002ffe-714c-4885-9805-352a82ba9988 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:53:57.379836 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 712/1406] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:53:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:53:59.832475 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64297ed3-a6bd-4235-8e9d-f71d02249f79 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:59.833052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64297ed3-a6bd-4235-8e9d-f71d02249f79 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:59.833052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64297ed3-a6bd-4235-8e9d-f71d02249f79 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:59.834371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 695/1407] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:53:59.839818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ed56b-1ca2-49b3-a2e5-02c609f2ec8c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:53:59.840261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ed56b-1ca2-49b3-a2e5-02c609f2ec8c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:53:59.840411 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ed56b-1ca2-49b3-a2e5-02c609f2ec8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:53:59.911995 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-4e9ed56b-1ca2-49b3-a2e5-02c609f2ec8c None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:53:59.913489 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4e9ed56b-1ca2-49b3-a2e5-02c609f2ec8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:53:59.965803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:00.025030 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 713/1408] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:53:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:54:00.324738 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64297ed3-a6bd-4235-8e9d-f71d02249f79 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:54:00.326239 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-64297ed3-a6bd-4235-8e9d-f71d02249f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:00.484709 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64297ed3-a6bd-4235-8e9d-f71d02249f79 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:54:00.559586 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:00.622156 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c236b-fece-439a-8a2e-b41ace458ec0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:54:00.622788 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c236b-fece-439a-8a2e-b41ace458ec0 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:00.623056 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c236b-fece-439a-8a2e-b41ace458ec0 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:00.623329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c236b-fece-439a-8a2e-b41ace458ec0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:00.625587 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-427c236b-fece-439a-8a2e-b41ace458ec0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:00.732741 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c236b-fece-439a-8a2e-b41ace458ec0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:54:00.734498 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c236b-fece-439a-8a2e-b41ace458ec0 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:54:00.767003 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 696/1409] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:54:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:54:10.025951 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=23 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:10.754576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=64 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:14.301879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163fccd-7b70-453d-8f72-ef9a51d23763 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:14.302406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163fccd-7b70-453d-8f72-ef9a51d23763 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:14.302406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163fccd-7b70-453d-8f72-ef9a51d23763 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:14.303805 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 714/1410] 10.222.0.17 () {54 vars in 918 bytes} [Sat Sep 10 23:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:54:14.309140 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40d109-49d0-4670-a16e-a4e85b6214f0 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:14.309561 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40d109-49d0-4670-a16e-a4e85b6214f0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:14.309687 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40d109-49d0-4670-a16e-a4e85b6214f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:14.386709 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-cf40d109-49d0-4670-a16e-a4e85b6214f0 None None] MFA Rules not processed for user `5cb1d85a338e40f18e61887a1becb906`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:54:14.387921 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-cf40d109-49d0-4670-a16e-a4e85b6214f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:14.497680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 697/1411] 10.222.0.17 () {58 vars in 1015 bytes} [Sat Sep 10 23:54:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:54:14.512530 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea48c1-a118-4070-b75e-59de4e2f08d5 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:14.512811 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea48c1-a118-4070-b75e-59de4e2f08d5 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:14.513084 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea48c1-a118-4070-b75e-59de4e2f08d5 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:14.514417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 715/1412] 10.222.0.17 () {54 vars in 918 bytes} [Sat Sep 10 23:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:54:14.520082 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0d6b0-b59c-4c79-825b-b6617e5d288a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:14.520362 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0d6b0-b59c-4c79-825b-b6617e5d288a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:14.520632 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0d6b0-b59c-4c79-825b-b6617e5d288a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:14.597966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:14.600247 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-e6b0d6b0-b59c-4c79-825b-b6617e5d288a None None] MFA Rules not processed for user `636bfafb693044d283530063047fc0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:54:14.601451 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e6b0d6b0-b59c-4c79-825b-b6617e5d288a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:14.717915 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 698/1413] 10.222.0.17 () {58 vars in 1015 bytes} [Sat Sep 10 23:54:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:54:17.426152 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf98f70-797c-4d1f-8b8d-c0c7e31af88a None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:17.426756 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf98f70-797c-4d1f-8b8d-c0c7e31af88a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:17.426756 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf98f70-797c-4d1f-8b8d-c0c7e31af88a None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:17.428078 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 716/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 23:54:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:54:17.434575 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6c27f-4f17-4767-be3a-77c72dfb37fd None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:17.435326 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6c27f-4f17-4767-be3a-77c72dfb37fd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:17.435326 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6c27f-4f17-4767-be3a-77c72dfb37fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:17.528097 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c4f6c27f-4f17-4767-be3a-77c72dfb37fd None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:54:17.529501 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c4f6c27f-4f17-4767-be3a-77c72dfb37fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:17.586017 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:17.635809 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 699/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 23:54:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:54:17.958084 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf98f70-797c-4d1f-8b8d-c0c7e31af88a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:54:17.959619 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1bf98f70-797c-4d1f-8b8d-c0c7e31af88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:18.221881 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf98f70-797c-4d1f-8b8d-c0c7e31af88a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:54:18.293723 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:18.335088 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d26a78-c143-40bf-815d-11cc21b279af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:54:18.335688 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d26a78-c143-40bf-815d-11cc21b279af service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:18.335955 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d26a78-c143-40bf-815d-11cc21b279af service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:18.336217 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d26a78-c143-40bf-815d-11cc21b279af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:18.337774 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-14d26a78-c143-40bf-815d-11cc21b279af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:18.427541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d26a78-c143-40bf-815d-11cc21b279af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:54:18.429138 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d26a78-c143-40bf-815d-11cc21b279af service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:54:18.447783 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 717/1416] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:54:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:54:19.598979 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d8f9e-4927-4dd6-97e7-3362087cd8af None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:19.599548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d8f9e-4927-4dd6-97e7-3362087cd8af None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:19.599548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d8f9e-4927-4dd6-97e7-3362087cd8af None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:19.600866 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 700/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 23:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:54:19.607053 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70578b2-1b20-4528-9475-e2f3cef83ba4 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:19.607457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70578b2-1b20-4528-9475-e2f3cef83ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:19.607602 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70578b2-1b20-4528-9475-e2f3cef83ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:19.683824 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-f70578b2-1b20-4528-9475-e2f3cef83ba4 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:54:19.685028 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f70578b2-1b20-4528-9475-e2f3cef83ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:19.711888 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:19.784329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 718/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 23:54:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:54:19.938557 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d8f9e-4927-4dd6-97e7-3362087cd8af None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:54:19.941438 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7a5d8f9e-4927-4dd6-97e7-3362087cd8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:20.083534 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d8f9e-4927-4dd6-97e7-3362087cd8af None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:54:20.122303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:20.194738 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633c82d-588e-4547-a8a3-863036a38f50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:54:20.195271 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633c82d-588e-4547-a8a3-863036a38f50 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:20.195548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633c82d-588e-4547-a8a3-863036a38f50 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:20.195818 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633c82d-588e-4547-a8a3-863036a38f50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:20.197409 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3633c82d-588e-4547-a8a3-863036a38f50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:20.285891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3633c82d-588e-4547-a8a3-863036a38f50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:54:20.287514 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3633c82d-588e-4547-a8a3-863036a38f50 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:54:20.306056 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 701/1419] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:54:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:54:21.430234 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ee616-42d1-4056-b411-5b10c716e735 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:21.430791 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ee616-42d1-4056-b411-5b10c716e735 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:21.430791 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ee616-42d1-4056-b411-5b10c716e735 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:21.432126 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 719/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 23:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:54:21.438577 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46b7e3-31d3-4afc-9013-74a9c9706b38 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:21.438996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46b7e3-31d3-4afc-9013-74a9c9706b38 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:21.439138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46b7e3-31d3-4afc-9013-74a9c9706b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:21.494289 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:21.515842 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5b46b7e3-31d3-4afc-9013-74a9c9706b38 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:54:21.517030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5b46b7e3-31d3-4afc-9013-74a9c9706b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:21.611955 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 702/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 23:54:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 23:54:21.764662 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ee616-42d1-4056-b411-5b10c716e735 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:54:21.766144 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ab9ee616-42d1-4056-b411-5b10c716e735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:21.910694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ee616-42d1-4056-b411-5b10c716e735 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:54:21.922503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:22.031440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e150f5c2-2144-4b7d-95e7-b570708eabeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:54:22.032036 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150f5c2-2144-4b7d-95e7-b570708eabeb service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:22.032309 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150f5c2-2144-4b7d-95e7-b570708eabeb service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:22.032573 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150f5c2-2144-4b7d-95e7-b570708eabeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:22.035318 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e150f5c2-2144-4b7d-95e7-b570708eabeb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:22.124733 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e150f5c2-2144-4b7d-95e7-b570708eabeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:54:22.126302 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e150f5c2-2144-4b7d-95e7-b570708eabeb service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:54:22.145181 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 720/1422] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:54:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:54:23.268606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553920e-a451-45f9-8a25-96683adbec01 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:23.269174 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553920e-a451-45f9-8a25-96683adbec01 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:23.269174 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553920e-a451-45f9-8a25-96683adbec01 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:23.270487 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 703/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 23:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:54:23.276898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e8136-b138-46b2-ad39-6a95813dc0d8 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:23.277335 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e8136-b138-46b2-ad39-6a95813dc0d8 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:23.277429 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e8136-b138-46b2-ad39-6a95813dc0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:23.297409 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:23.356586 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-451e8136-b138-46b2-ad39-6a95813dc0d8 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:54:23.357779 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-451e8136-b138-46b2-ad39-6a95813dc0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:23.450951 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 721/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 23:54:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:54:23.603901 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553920e-a451-45f9-8a25-96683adbec01 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:54:23.605325 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0553920e-a451-45f9-8a25-96683adbec01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:23.714343 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:23.749706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553920e-a451-45f9-8a25-96683adbec01 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:54:23.866961 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae655ab-5824-4d33-9334-be1653307201 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:54:23.867470 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae655ab-5824-4d33-9334-be1653307201 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:23.867738 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae655ab-5824-4d33-9334-be1653307201 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:23.867997 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae655ab-5824-4d33-9334-be1653307201 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:23.869569 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5ae655ab-5824-4d33-9334-be1653307201 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:23.946808 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:23.960643 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae655ab-5824-4d33-9334-be1653307201 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:54:23.962266 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae655ab-5824-4d33-9334-be1653307201 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:54:23.980437 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 704/1425] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:54:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:54:25.116437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828b8b1-8a39-4f14-92d2-0c0071d16e84 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:25.117054 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828b8b1-8a39-4f14-92d2-0c0071d16e84 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:25.117054 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828b8b1-8a39-4f14-92d2-0c0071d16e84 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:25.118293 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 722/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 23:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:54:25.124497 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4692d46-dd07-4dca-8c62-8ce2201fa8dd None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:25.124961 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4692d46-dd07-4dca-8c62-8ce2201fa8dd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:25.125068 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4692d46-dd07-4dca-8c62-8ce2201fa8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:25.200542 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f4692d46-dd07-4dca-8c62-8ce2201fa8dd None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:54:25.201752 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f4692d46-dd07-4dca-8c62-8ce2201fa8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:25.303359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 705/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 23:54:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:54:25.451436 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828b8b1-8a39-4f14-92d2-0c0071d16e84 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:54:25.453022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8828b8b1-8a39-4f14-92d2-0c0071d16e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:25.537978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:25.605689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828b8b1-8a39-4f14-92d2-0c0071d16e84 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:54:25.720253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf0365-0e6a-4732-a243-0189c8c0cfff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:54:25.720967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf0365-0e6a-4732-a243-0189c8c0cfff service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:25.721266 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf0365-0e6a-4732-a243-0189c8c0cfff service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:25.721536 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf0365-0e6a-4732-a243-0189c8c0cfff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:25.723145 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-efcf0365-0e6a-4732-a243-0189c8c0cfff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:25.771887 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:25.812991 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf0365-0e6a-4732-a243-0189c8c0cfff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:54:25.815341 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf0365-0e6a-4732-a243-0189c8c0cfff service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:54:25.833338 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 723/1428] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:54:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:54:26.969910 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef560c-def2-4cd1-bb24-c9d8fa30f9a5 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:26.970486 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef560c-def2-4cd1-bb24-c9d8fa30f9a5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:26.970486 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef560c-def2-4cd1-bb24-c9d8fa30f9a5 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:26.971943 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 706/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 23:54:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:54:26.978129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5320e0f-e9b1-427b-8158-f6e69d866f54 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:26.978566 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5320e0f-e9b1-427b-8158-f6e69d866f54 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:26.978687 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5320e0f-e9b1-427b-8158-f6e69d866f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:27.053255 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-f5320e0f-e9b1-427b-8158-f6e69d866f54 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:54:27.054435 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f5320e0f-e9b1-427b-8158-f6e69d866f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:27.147331 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 724/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 23:54:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:54:27.299982 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef560c-def2-4cd1-bb24-c9d8fa30f9a5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:54:27.301534 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e9ef560c-def2-4cd1-bb24-c9d8fa30f9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:27.350741 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:27.445175 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef560c-def2-4cd1-bb24-c9d8fa30f9a5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:54:27.554390 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555fe5a7-ae2c-4898-a283-6a156549c4e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:54:27.554947 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fe5a7-ae2c-4898-a283-6a156549c4e2 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:27.555224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fe5a7-ae2c-4898-a283-6a156549c4e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:27.555488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fe5a7-ae2c-4898-a283-6a156549c4e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:27.557055 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-555fe5a7-ae2c-4898-a283-6a156549c4e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:27.563785 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:27.647377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555fe5a7-ae2c-4898-a283-6a156549c4e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:54:27.649111 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555fe5a7-ae2c-4898-a283-6a156549c4e2 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:54:27.667164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 707/1431] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:54:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:54:28.784708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf4021-2ecf-4aad-abf3-ad1468361d31 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:28.785286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf4021-2ecf-4aad-abf3-ad1468361d31 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:28.785286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf4021-2ecf-4aad-abf3-ad1468361d31 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:28.786624 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 725/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 23:54:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:54:28.792937 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23801525-9b1b-4e8a-842d-f5a0889bbfc9 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:28.793373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23801525-9b1b-4e8a-842d-f5a0889bbfc9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:28.793485 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23801525-9b1b-4e8a-842d-f5a0889bbfc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:28.868723 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-23801525-9b1b-4e8a-842d-f5a0889bbfc9 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:54:28.869909 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-23801525-9b1b-4e8a-842d-f5a0889bbfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:28.937729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:28.966022 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 708/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 23:54:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:54:29.116880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf4021-2ecf-4aad-abf3-ad1468361d31 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:54:29.118312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4aaf4021-2ecf-4aad-abf3-ad1468361d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:29.135960 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:29.262000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf4021-2ecf-4aad-abf3-ad1468361d31 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:54:29.349376 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:29.375051 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0454026-c408-4cf8-82dd-6b4e57fa0364 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:54:29.375599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0454026-c408-4cf8-82dd-6b4e57fa0364 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:29.375875 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0454026-c408-4cf8-82dd-6b4e57fa0364 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:29.376143 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0454026-c408-4cf8-82dd-6b4e57fa0364 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:29.378440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e0454026-c408-4cf8-82dd-6b4e57fa0364 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:29.467447 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0454026-c408-4cf8-82dd-6b4e57fa0364 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:54:29.469046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0454026-c408-4cf8-82dd-6b4e57fa0364 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:54:29.487101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 726/1434] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:54:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:54:30.588875 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee9eda-19f2-4b9e-b367-7683ceb772ae None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:30.589421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee9eda-19f2-4b9e-b367-7683ceb772ae None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:30.589421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee9eda-19f2-4b9e-b367-7683ceb772ae None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:30.590713 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 709/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 23:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:54:30.597257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a3850-06ed-4479-b70a-c907472032f3 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:30.597684 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a3850-06ed-4479-b70a-c907472032f3 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:30.597774 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a3850-06ed-4479-b70a-c907472032f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:30.671924 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-432a3850-06ed-4479-b70a-c907472032f3 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:54:30.673113 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-432a3850-06ed-4479-b70a-c907472032f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:30.714557 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:30.772514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 727/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 23:54:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:54:30.922202 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee9eda-19f2-4b9e-b367-7683ceb772ae None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:54:30.924265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-77ee9eda-19f2-4b9e-b367-7683ceb772ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:31.068114 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee9eda-19f2-4b9e-b367-7683ceb772ae None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:54:31.120521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:31.179153 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77158d1-23a6-47f1-a52a-56cafc4aea6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:54:31.179674 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77158d1-23a6-47f1-a52a-56cafc4aea6c service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:31.179946 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77158d1-23a6-47f1-a52a-56cafc4aea6c service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:31.180208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77158d1-23a6-47f1-a52a-56cafc4aea6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:31.181779 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f77158d1-23a6-47f1-a52a-56cafc4aea6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:31.270261 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77158d1-23a6-47f1-a52a-56cafc4aea6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:54:31.271894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77158d1-23a6-47f1-a52a-56cafc4aea6c service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:54:31.289335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 710/1437] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:54:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:54:32.423041 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73e402-13f1-4685-84ae-c386c2c400d7 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:32.431142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73e402-13f1-4685-84ae-c386c2c400d7 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:32.431142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73e402-13f1-4685-84ae-c386c2c400d7 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:32.431142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 728/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 23:54:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:54:32.431974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a54400-2975-4279-9697-69e8dbf016bd None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:32.432371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a54400-2975-4279-9697-69e8dbf016bd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:32.432519 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a54400-2975-4279-9697-69e8dbf016bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:32.511300 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-78a54400-2975-4279-9697-69e8dbf016bd None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:54:32.512484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-78a54400-2975-4279-9697-69e8dbf016bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:32.514438 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:32.606652 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 711/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 23:54:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:54:32.763470 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc73e402-13f1-4685-84ae-c386c2c400d7 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:54:32.764987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-bc73e402-13f1-4685-84ae-c386c2c400d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:32.908795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc73e402-13f1-4685-84ae-c386c2c400d7 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:54:32.935451 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:33.020178 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a04635-1836-4815-aa12-abf8c40f0869 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:54:33.020814 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a04635-1836-4815-aa12-abf8c40f0869 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:33.021091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a04635-1836-4815-aa12-abf8c40f0869 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:33.021365 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a04635-1836-4815-aa12-abf8c40f0869 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:33.022986 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-76a04635-1836-4815-aa12-abf8c40f0869 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:33.112170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a04635-1836-4815-aa12-abf8c40f0869 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:54:33.113780 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a04635-1836-4815-aa12-abf8c40f0869 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:54:33.132100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 729/1440] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:54:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:54:35.196661 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78928ae9-1b8a-4813-951f-fa62fd4ee919 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:35.197234 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78928ae9-1b8a-4813-951f-fa62fd4ee919 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:35.197234 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78928ae9-1b8a-4813-951f-fa62fd4ee919 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:35.198574 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 712/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 23:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:54:35.203774 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e945f0-adcf-403c-aa56-118239a98e5b None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:35.204211 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e945f0-adcf-403c-aa56-118239a98e5b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:35.204316 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e945f0-adcf-403c-aa56-118239a98e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:35.244741 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:35.269857 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b3e945f0-adcf-403c-aa56-118239a98e5b None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:54:35.271080 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b3e945f0-adcf-403c-aa56-118239a98e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:35.368204 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 730/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 23:54:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:54:35.654803 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78928ae9-1b8a-4813-951f-fa62fd4ee919 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:54:35.656354 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-78928ae9-1b8a-4813-951f-fa62fd4ee919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:35.758588 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78928ae9-1b8a-4813-951f-fa62fd4ee919 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:54:35.790501 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:35.833021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aacfa0-dc87-42ae-860d-22598aded2f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:54:35.833613 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aacfa0-dc87-42ae-860d-22598aded2f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:35.833885 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aacfa0-dc87-42ae-860d-22598aded2f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:35.834149 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aacfa0-dc87-42ae-860d-22598aded2f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:35.835746 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e4aacfa0-dc87-42ae-860d-22598aded2f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:35.947001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aacfa0-dc87-42ae-860d-22598aded2f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:54:35.948447 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aacfa0-dc87-42ae-860d-22598aded2f9 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:54:35.967039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 713/1443] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:54:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:54:38.299327 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6eaf7-6a4b-47b8-88fc-7a744ee4b873 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:38.299937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6eaf7-6a4b-47b8-88fc-7a744ee4b873 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:38.300044 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6eaf7-6a4b-47b8-88fc-7a744ee4b873 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:38.301427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 731/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 23:54:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:54:38.307283 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994a307-fc6e-4d28-8e0b-52fd0c7c3cd1 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:38.307283 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994a307-fc6e-4d28-8e0b-52fd0c7c3cd1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:38.307747 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994a307-fc6e-4d28-8e0b-52fd0c7c3cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:38.372919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-e994a307-fc6e-4d28-8e0b-52fd0c7c3cd1 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:54:38.374110 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e994a307-fc6e-4d28-8e0b-52fd0c7c3cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:38.386945 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:38.471725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 714/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 23:54:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:54:38.769051 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b6eaf7-6a4b-47b8-88fc-7a744ee4b873 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:54:38.770567 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-51b6eaf7-6a4b-47b8-88fc-7a744ee4b873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:38.871285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b6eaf7-6a4b-47b8-88fc-7a744ee4b873 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:54:38.883829 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:38.944794 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40efbc-0337-4b2d-8d5d-75d5732c937b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:54:38.945386 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40efbc-0337-4b2d-8d5d-75d5732c937b service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:38.945659 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40efbc-0337-4b2d-8d5d-75d5732c937b service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:38.945923 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40efbc-0337-4b2d-8d5d-75d5732c937b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:38.947539 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8e40efbc-0337-4b2d-8d5d-75d5732c937b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:39.056647 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40efbc-0337-4b2d-8d5d-75d5732c937b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:54:39.058001 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40efbc-0337-4b2d-8d5d-75d5732c937b service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:54:39.076300 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 732/1446] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:54:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:54:41.616421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d64fd17-eaaf-4ce7-bc6f-ecde5f4348f2 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:41.616958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d64fd17-eaaf-4ce7-bc6f-ecde5f4348f2 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:41.616958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d64fd17-eaaf-4ce7-bc6f-ecde5f4348f2 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:41.618555 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 715/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 23:54:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:54:41.623936 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44947614-c812-40f9-87fe-ea6770654c2a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:41.624363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44947614-c812-40f9-87fe-ea6770654c2a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:41.624457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44947614-c812-40f9-87fe-ea6770654c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:41.671755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:41.689829 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-44947614-c812-40f9-87fe-ea6770654c2a None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:54:41.691686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-44947614-c812-40f9-87fe-ea6770654c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:41.787251 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 733/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 23:54:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:54:42.078126 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d64fd17-eaaf-4ce7-bc6f-ecde5f4348f2 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:54:42.080260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8d64fd17-eaaf-4ce7-bc6f-ecde5f4348f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:42.182763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d64fd17-eaaf-4ce7-bc6f-ecde5f4348f2 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:54:42.223973 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:42.258109 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2350d8f0-41d1-43e6-aea7-a26732894c13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:54:42.258727 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350d8f0-41d1-43e6-aea7-a26732894c13 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:42.259005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350d8f0-41d1-43e6-aea7-a26732894c13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:42.259273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350d8f0-41d1-43e6-aea7-a26732894c13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:42.260838 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2350d8f0-41d1-43e6-aea7-a26732894c13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:42.378376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2350d8f0-41d1-43e6-aea7-a26732894c13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:54:42.378376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2350d8f0-41d1-43e6-aea7-a26732894c13 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:54:42.393636 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 716/1449] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:54:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:54:44.684223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d4f55-2963-4ee8-a7bb-b78592b243c2 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:44.684852 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d4f55-2963-4ee8-a7bb-b78592b243c2 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:44.684852 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d4f55-2963-4ee8-a7bb-b78592b243c2 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:44.686129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 734/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 23:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:54:44.692282 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea26b3-3426-4dee-ab72-f94adc03092b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:44.692734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea26b3-3426-4dee-ab72-f94adc03092b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:44.692833 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea26b3-3426-4dee-ab72-f94adc03092b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:44.759348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d8ea26b3-3426-4dee-ab72-f94adc03092b None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:54:44.760543 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d8ea26b3-3426-4dee-ab72-f94adc03092b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:44.780753 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:44.862219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 717/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 23:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:54:45.148744 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d4f55-2963-4ee8-a7bb-b78592b243c2 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:54:45.150285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-df4d4f55-2963-4ee8-a7bb-b78592b243c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:45.253740 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d4f55-2963-4ee8-a7bb-b78592b243c2 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:54:45.274368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:45.328286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de218dc-df3b-445d-aa73-29703f2b989f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:54:45.328881 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de218dc-df3b-445d-aa73-29703f2b989f service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:45.329150 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de218dc-df3b-445d-aa73-29703f2b989f service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:45.329989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de218dc-df3b-445d-aa73-29703f2b989f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:45.331546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2de218dc-df3b-445d-aa73-29703f2b989f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:45.441925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de218dc-df3b-445d-aa73-29703f2b989f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:54:45.443383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de218dc-df3b-445d-aa73-29703f2b989f service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:54:45.462191 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 735/1452] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:54:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:54:47.687784 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10f580-a4d0-4278-aeb8-72d8a3cd0c0f None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:47.688313 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10f580-a4d0-4278-aeb8-72d8a3cd0c0f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:47.688313 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10f580-a4d0-4278-aeb8-72d8a3cd0c0f None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:47.690451 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 718/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 23:54:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:54:47.696258 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee4a24-42ae-4a79-bf2f-1726dcfbe505 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:47.696664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee4a24-42ae-4a79-bf2f-1726dcfbe505 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:47.696797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee4a24-42ae-4a79-bf2f-1726dcfbe505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:47.759366 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:47.761995 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-00ee4a24-42ae-4a79-bf2f-1726dcfbe505 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:54:47.763239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-00ee4a24-42ae-4a79-bf2f-1726dcfbe505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:47.858273 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 736/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 23:54:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:54:48.150508 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a10f580-a4d0-4278-aeb8-72d8a3cd0c0f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:54:48.152067 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7a10f580-a4d0-4278-aeb8-72d8a3cd0c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:48.255134 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a10f580-a4d0-4278-aeb8-72d8a3cd0c0f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:54:48.305239 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:48.329244 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975cd8b-5a0d-49d8-8f0a-5fbf6c70593e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:54:48.330519 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975cd8b-5a0d-49d8-8f0a-5fbf6c70593e service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:48.330771 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975cd8b-5a0d-49d8-8f0a-5fbf6c70593e service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:48.331040 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975cd8b-5a0d-49d8-8f0a-5fbf6c70593e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:48.332611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1975cd8b-5a0d-49d8-8f0a-5fbf6c70593e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:48.440591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975cd8b-5a0d-49d8-8f0a-5fbf6c70593e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:54:48.442464 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975cd8b-5a0d-49d8-8f0a-5fbf6c70593e service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:54:48.461439 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 719/1455] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:54:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:54:50.677908 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acfbd1-00b3-4336-ba89-7f92cce3f11c None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:50.678432 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acfbd1-00b3-4336-ba89-7f92cce3f11c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:50.678432 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acfbd1-00b3-4336-ba89-7f92cce3f11c None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:50.679831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 737/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 23:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:54:50.685087 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98c73d-8ee6-47cc-b52e-6c0ef1698761 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:50.685488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98c73d-8ee6-47cc-b52e-6c0ef1698761 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:50.685657 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98c73d-8ee6-47cc-b52e-6c0ef1698761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:50.753873 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-bb98c73d-8ee6-47cc-b52e-6c0ef1698761 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:54:50.755117 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-bb98c73d-8ee6-47cc-b52e-6c0ef1698761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:50.783459 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:50.852466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 720/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 23:54:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:54:51.141502 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76acfbd1-00b3-4336-ba89-7f92cce3f11c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:54:51.143605 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-76acfbd1-00b3-4336-ba89-7f92cce3f11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:51.245237 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76acfbd1-00b3-4336-ba89-7f92cce3f11c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:54:51.273586 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:51.319451 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dca249-a38e-491c-b0ac-d41d14a466b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:54:51.320065 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dca249-a38e-491c-b0ac-d41d14a466b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:51.320340 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dca249-a38e-491c-b0ac-d41d14a466b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:51.320611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dca249-a38e-491c-b0ac-d41d14a466b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:51.322198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-59dca249-a38e-491c-b0ac-d41d14a466b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:51.429439 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dca249-a38e-491c-b0ac-d41d14a466b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:54:51.430902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dca249-a38e-491c-b0ac-d41d14a466b9 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:54:51.449315 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 738/1458] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:54:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:54:53.665572 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2867b-cbff-415e-b9e5-96cb006ff329 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:53.666113 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2867b-cbff-415e-b9e5-96cb006ff329 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:53.666113 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2867b-cbff-415e-b9e5-96cb006ff329 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:53.667512 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 721/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 23:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:54:53.672862 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947da58-342e-466f-b9cc-6cbf709db540 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:53.674133 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947da58-342e-466f-b9cc-6cbf709db540 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:53.674133 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947da58-342e-466f-b9cc-6cbf709db540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:53.740403 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-d947da58-342e-466f-b9cc-6cbf709db540 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:54:53.741593 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-d947da58-342e-466f-b9cc-6cbf709db540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:53.749680 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:53.836140 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 739/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 23:54:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:54:54.124935 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2867b-cbff-415e-b9e5-96cb006ff329 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:54:54.126511 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-35f2867b-cbff-415e-b9e5-96cb006ff329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:54.230033 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2867b-cbff-415e-b9e5-96cb006ff329 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:54:54.284622 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:54.303446 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f170d92-adfb-4225-b3a7-2ab722d5f849 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:54:54.306724 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f170d92-adfb-4225-b3a7-2ab722d5f849 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:54.306724 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f170d92-adfb-4225-b3a7-2ab722d5f849 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:54.306724 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f170d92-adfb-4225-b3a7-2ab722d5f849 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:54.306724 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3f170d92-adfb-4225-b3a7-2ab722d5f849 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:54.417259 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f170d92-adfb-4225-b3a7-2ab722d5f849 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:54:54.428409 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f170d92-adfb-4225-b3a7-2ab722d5f849 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:54:54.437128 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 722/1461] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:54:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:54:56.734520 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df8952-4dec-410f-a87c-118e3c4d7fd1 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:56.735184 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df8952-4dec-410f-a87c-118e3c4d7fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:56.735184 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df8952-4dec-410f-a87c-118e3c4d7fd1 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:56.736756 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 740/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 23:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:54:56.742009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecefd89-b120-4f55-b367-4ba652df23e3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:56.742438 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecefd89-b120-4f55-b367-4ba652df23e3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:56.742539 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecefd89-b120-4f55-b367-4ba652df23e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:56.807496 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-eecefd89-b120-4f55-b367-4ba652df23e3 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:54:56.808706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-eecefd89-b120-4f55-b367-4ba652df23e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:56.847931 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:56.906001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 723/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 23:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 23:54:57.193341 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df8952-4dec-410f-a87c-118e3c4d7fd1 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:54:57.194895 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f6df8952-4dec-410f-a87c-118e3c4d7fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:57.295039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df8952-4dec-410f-a87c-118e3c4d7fd1 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:54:57.334585 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:57.369883 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20504e12-bd2f-4b1e-98f3-4cb8cf7b640c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:54:57.370730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20504e12-bd2f-4b1e-98f3-4cb8cf7b640c service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:57.371069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20504e12-bd2f-4b1e-98f3-4cb8cf7b640c service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:57.371530 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20504e12-bd2f-4b1e-98f3-4cb8cf7b640c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:57.373539 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-20504e12-bd2f-4b1e-98f3-4cb8cf7b640c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:57.481071 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20504e12-bd2f-4b1e-98f3-4cb8cf7b640c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:54:57.482489 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20504e12-bd2f-4b1e-98f3-4cb8cf7b640c service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:54:57.501482 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 741/1464] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:54:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:54:59.711788 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9daba64-8b81-4969-97ad-0618c5e94782 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:59.712354 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9daba64-8b81-4969-97ad-0618c5e94782 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:59.712468 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9daba64-8b81-4969-97ad-0618c5e94782 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:59.713749 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 724/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 23:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:54:59.719056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256db805-7e71-4979-a5c5-e941afbf58ae None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:54:59.719502 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256db805-7e71-4979-a5c5-e941afbf58ae None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:54:59.719598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256db805-7e71-4979-a5c5-e941afbf58ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:54:59.786151 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-256db805-7e71-4979-a5c5-e941afbf58ae None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:54:59.787401 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-256db805-7e71-4979-a5c5-e941afbf58ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:54:59.805060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:54:59.883439 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 742/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 23:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:55:00.169696 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9daba64-8b81-4969-97ad-0618c5e94782 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:55:00.171235 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e9daba64-8b81-4969-97ad-0618c5e94782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:55:00.273010 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9daba64-8b81-4969-97ad-0618c5e94782 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:55:00.340150 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:55:00.347127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f46847-2f20-4166-af76-8acf81309027 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:55:00.355479 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f46847-2f20-4166-af76-8acf81309027 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:55:00.355479 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f46847-2f20-4166-af76-8acf81309027 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:55:00.355479 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f46847-2f20-4166-af76-8acf81309027 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:55:00.355479 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-90f46847-2f20-4166-af76-8acf81309027 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:55:00.458963 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f46847-2f20-4166-af76-8acf81309027 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:55:00.460350 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f46847-2f20-4166-af76-8acf81309027 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:55:00.478310 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 725/1467] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:55:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:55:02.676465 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669aff2e-c37a-4c31-a355-1e09bd9adb85 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:55:02.677010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669aff2e-c37a-4c31-a355-1e09bd9adb85 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:55:02.677010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669aff2e-c37a-4c31-a355-1e09bd9adb85 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:55:02.678329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 743/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 23:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:55:02.683695 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592e979-ca8e-4272-b4a8-80fcbca95f2f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:55:02.684133 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592e979-ca8e-4272-b4a8-80fcbca95f2f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:55:02.684236 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592e979-ca8e-4272-b4a8-80fcbca95f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:55:02.749817 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f592e979-ca8e-4272-b4a8-80fcbca95f2f None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:55:02.752237 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f592e979-ca8e-4272-b4a8-80fcbca95f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:55:02.800122 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:55:02.848799 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 726/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 23:55:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:55:03.138566 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669aff2e-c37a-4c31-a355-1e09bd9adb85 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:55:03.140125 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-669aff2e-c37a-4c31-a355-1e09bd9adb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:55:03.242192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669aff2e-c37a-4c31-a355-1e09bd9adb85 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:55:03.289174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:55:03.316172 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bc72a-e807-4b8f-bde0-9f6db201bdcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:55:03.316780 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bc72a-e807-4b8f-bde0-9f6db201bdcb service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:55:03.317057 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bc72a-e807-4b8f-bde0-9f6db201bdcb service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:55:03.317327 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bc72a-e807-4b8f-bde0-9f6db201bdcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:55:03.319692 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6a7bc72a-e807-4b8f-bde0-9f6db201bdcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:55:03.426627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7bc72a-e807-4b8f-bde0-9f6db201bdcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:55:03.428069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7bc72a-e807-4b8f-bde0-9f6db201bdcb service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:55:03.446425 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 744/1470] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:55:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:55:12.847286 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=22 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:55:13.444158 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=66 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:55:23.807682 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3f265-6a11-47c7-8fdb-00f9091abf84 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:55:23.808243 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3f265-6a11-47c7-8fdb-00f9091abf84 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:55:23.808243 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3f265-6a11-47c7-8fdb-00f9091abf84 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:55:23.809571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 727/1471] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:55:23.815365 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ddca5-e816-43f9-96d4-ff79f9dd8ef2 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:55:23.815769 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ddca5-e816-43f9-96d4-ff79f9dd8ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:55:23.815898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ddca5-e816-43f9-96d4-ff79f9dd8ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:55:23.881112 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-662ddca5-e816-43f9-96d4-ff79f9dd8ef2 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:55:23.882320 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-662ddca5-e816-43f9-96d4-ff79f9dd8ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:55:23.987626 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 745/1472] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:55:24.295625 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d2714-eba3-4473-9420-a70abb66620d None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:55:24.295908 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d2714-eba3-4473-9420-a70abb66620d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:55:24.296183 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d2714-eba3-4473-9420-a70abb66620d None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:55:24.297520 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 728/1473] 10.222.0.17 () {54 vars in 966 bytes} [Sat Sep 10 23:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:55:24.303076 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a444d3e-92c3-4f59-8c68-7e67dea8a9e4 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:55:24.303359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a444d3e-92c3-4f59-8c68-7e67dea8a9e4 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:55:24.303625 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a444d3e-92c3-4f59-8c68-7e67dea8a9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:55:24.380470 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-6a444d3e-92c3-4f59-8c68-7e67dea8a9e4 None None] MFA Rules not processed for user `2cb18199dc8b42589184042556e98983`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:55:24.381696 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6a444d3e-92c3-4f59-8c68-7e67dea8a9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:55:24.403152 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:55:24.458627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 746/1474] 10.222.0.17 () {58 vars in 1063 bytes} [Sat Sep 10 23:55:24 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:55:24.466628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d2714-eba3-4473-9420-a70abb66620d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:55:24.468317 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d32d2714-eba3-4473-9420-a70abb66620d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:55:24.575327 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d2714-eba3-4473-9420-a70abb66620d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:55:24.647696 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fca520-280b-4d31-9910-e7676e9c2b34 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb18199dc8b42589184042556e98983', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:55:24.648322 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fca520-280b-4d31-9910-e7676e9c2b34 service glance] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:55:24.648597 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fca520-280b-4d31-9910-e7676e9c2b34 service glance] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:55:24.648865 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fca520-280b-4d31-9910-e7676e9c2b34 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:55:24.650457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a7fca520-280b-4d31-9910-e7676e9c2b34 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:55:24.706979 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:55:24.761537 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fca520-280b-4d31-9910-e7676e9c2b34 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:55:24.763015 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fca520-280b-4d31-9910-e7676e9c2b34 service glance] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:55:24.781324 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 729/1475] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:55:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:55:27.041996 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876682a-019b-4b7f-9013-3357ac63a6dd None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:55:27.042607 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876682a-019b-4b7f-9013-3357ac63a6dd None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:55:27.042607 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876682a-019b-4b7f-9013-3357ac63a6dd None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:55:27.043943 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 747/1476] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:55:27.049263 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9817f-0534-417b-b9cb-c2b6e496eae5 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:55:27.049719 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9817f-0534-417b-b9cb-c2b6e496eae5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:55:27.049819 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a9817f-0534-417b-b9cb-c2b6e496eae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:55:27.117476 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-93a9817f-0534-417b-b9cb-c2b6e496eae5 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:55:27.118707 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-93a9817f-0534-417b-b9cb-c2b6e496eae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:55:27.213234 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 730/1477] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:55:27.490405 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1876682a-019b-4b7f-9013-3357ac63a6dd None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:55:27.492021 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1876682a-019b-4b7f-9013-3357ac63a6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:55:27.618795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1876682a-019b-4b7f-9013-3357ac63a6dd None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:55:27.667699 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:55:27.712549 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10bc405-d6fa-4089-a6fb-0b882bcf17da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:55:27.713148 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bc405-d6fa-4089-a6fb-0b882bcf17da admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:55:27.713420 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bc405-d6fa-4089-a6fb-0b882bcf17da admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:55:27.713685 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bc405-d6fa-4089-a6fb-0b882bcf17da admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:55:27.714313 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10bc405-d6fa-4089-a6fb-0b882bcf17da admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:55:27.716037 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: WARNING py.warnings [None req-e10bc405-d6fa-4089-a6fb-0b882bcf17da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 23:55:27.716037 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: warnings.warn(msg) Sep 10 23:55:27.716037 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: Sep 10 23:55:27.717178 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10bc405-d6fa-4089-a6fb-0b882bcf17da admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:55:27.737129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 748/1478] 10.222.0.17 () {56 vars in 1108 bytes} [Sat Sep 10 23:55:27 2022] GET /identity/v3/projects => generated 2553 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 23:55:29.888785 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94339b-0308-4ae0-a378-6fe807df6721 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:55:29.889363 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94339b-0308-4ae0-a378-6fe807df6721 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:55:29.889363 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94339b-0308-4ae0-a378-6fe807df6721 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:55:29.890751 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 731/1479] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:55:29.896304 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ae2a9-b03c-4dae-8cc2-b73c88698a75 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:55:29.896775 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ae2a9-b03c-4dae-8cc2-b73c88698a75 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:55:29.896884 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ae2a9-b03c-4dae-8cc2-b73c88698a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:55:29.971296 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b46ae2a9-b03c-4dae-8cc2-b73c88698a75 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:55:29.972500 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b46ae2a9-b03c-4dae-8cc2-b73c88698a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:55:30.068762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 749/1480] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:55:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:55:30.364461 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94339b-0308-4ae0-a378-6fe807df6721 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:55:30.365965 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7e94339b-0308-4ae0-a378-6fe807df6721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:55:30.375662 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:55:30.510238 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94339b-0308-4ae0-a378-6fe807df6721 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:55:30.589280 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:55:30.621640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f260e17-cf51-4433-8862-7afdf7c02a18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:55:30.622252 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f260e17-cf51-4433-8862-7afdf7c02a18 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:55:30.622834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f260e17-cf51-4433-8862-7afdf7c02a18 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:55:30.623110 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f260e17-cf51-4433-8862-7afdf7c02a18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:55:30.624702 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4f260e17-cf51-4433-8862-7afdf7c02a18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:55:30.717172 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f260e17-cf51-4433-8862-7afdf7c02a18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:55:30.718828 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f260e17-cf51-4433-8862-7afdf7c02a18 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:55:30.738893 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 732/1481] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:55:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:55:33.117297 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804fdd72-2d48-4205-9195-e206d8049741 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:55:33.117883 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804fdd72-2d48-4205-9195-e206d8049741 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:55:33.117883 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804fdd72-2d48-4205-9195-e206d8049741 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:55:33.119198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 750/1482] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:55:33.124521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364f115-f207-4fa6-b926-bcded47b190c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:55:33.124924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364f115-f207-4fa6-b926-bcded47b190c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:55:33.125057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364f115-f207-4fa6-b926-bcded47b190c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:55:33.190310 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5364f115-f207-4fa6-b926-bcded47b190c None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:55:33.191769 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5364f115-f207-4fa6-b926-bcded47b190c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:55:33.237960 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:55:33.288594 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 733/1483] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:55:33.580125 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804fdd72-2d48-4205-9195-e206d8049741 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:55:33.581691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-804fdd72-2d48-4205-9195-e206d8049741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:55:33.605659 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:55:33.685557 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804fdd72-2d48-4205-9195-e206d8049741 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:55:33.758396 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f60c4-8121-48c1-9c88-732cbda2883a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:55:33.759018 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f60c4-8121-48c1-9c88-732cbda2883a service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:55:33.759294 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f60c4-8121-48c1-9c88-732cbda2883a service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:55:33.759558 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f60c4-8121-48c1-9c88-732cbda2883a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:55:33.761127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-bb3f60c4-8121-48c1-9c88-732cbda2883a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:55:33.833820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:55:33.867892 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f60c4-8121-48c1-9c88-732cbda2883a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:55:33.869320 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f60c4-8121-48c1-9c88-732cbda2883a service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:55:33.887639 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 751/1484] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:55:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:55:36.087381 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef4d15-4437-4822-86ac-cee13727da58 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:55:36.087927 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef4d15-4437-4822-86ac-cee13727da58 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:55:36.087927 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef4d15-4437-4822-86ac-cee13727da58 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:55:36.089264 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 734/1485] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:55:36.094560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39564ef5-f8fa-4278-a311-c068eba6057d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:55:36.096132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39564ef5-f8fa-4278-a311-c068eba6057d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:55:36.096132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39564ef5-f8fa-4278-a311-c068eba6057d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:55:36.161707 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-39564ef5-f8fa-4278-a311-c068eba6057d None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:55:36.162949 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-39564ef5-f8fa-4278-a311-c068eba6057d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:55:36.257138 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 752/1486] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:55:36.550406 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef4d15-4437-4822-86ac-cee13727da58 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:55:36.552053 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1aef4d15-4437-4822-86ac-cee13727da58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:55:36.655821 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef4d15-4437-4822-86ac-cee13727da58 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:55:36.727508 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:55:36.728990 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00378aae-a2b6-45b3-bb97-8352315c1f47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:55:36.729586 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00378aae-a2b6-45b3-bb97-8352315c1f47 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:55:36.729862 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00378aae-a2b6-45b3-bb97-8352315c1f47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:55:36.730133 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00378aae-a2b6-45b3-bb97-8352315c1f47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:55:36.731763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-00378aae-a2b6-45b3-bb97-8352315c1f47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:55:36.839847 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00378aae-a2b6-45b3-bb97-8352315c1f47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:55:36.841203 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00378aae-a2b6-45b3-bb97-8352315c1f47 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:55:36.858609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 735/1487] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:55:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:55:39.155161 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9c75e-406a-46de-a601-a93a42664c7c None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:55:39.155741 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9c75e-406a-46de-a601-a93a42664c7c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:55:39.155741 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9c75e-406a-46de-a601-a93a42664c7c None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:55:39.157671 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 753/1488] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:55:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:55:39.163121 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2064055-30df-4eb5-b8c3-a2cdd8eb4dfb None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:55:39.163546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2064055-30df-4eb5-b8c3-a2cdd8eb4dfb None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:55:39.163672 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2064055-30df-4eb5-b8c3-a2cdd8eb4dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:55:39.228865 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b2064055-30df-4eb5-b8c3-a2cdd8eb4dfb None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:55:39.230044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b2064055-30df-4eb5-b8c3-a2cdd8eb4dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:55:39.287186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:55:39.331239 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 736/1489] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:55:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:55:39.634563 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9c75e-406a-46de-a601-a93a42664c7c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:55:39.636110 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cdd9c75e-406a-46de-a601-a93a42664c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:55:39.675128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:55:39.743262 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9c75e-406a-46de-a601-a93a42664c7c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:55:39.817641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0a74d-fc44-4214-9a27-3666a90ba5c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:55:39.818353 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0a74d-fc44-4214-9a27-3666a90ba5c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:55:39.818734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0a74d-fc44-4214-9a27-3666a90ba5c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:55:39.819001 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0a74d-fc44-4214-9a27-3666a90ba5c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:55:39.820628 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-def0a74d-fc44-4214-9a27-3666a90ba5c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:55:39.900827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:55:39.930225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0a74d-fc44-4214-9a27-3666a90ba5c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:55:39.931716 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0a74d-fc44-4214-9a27-3666a90ba5c3 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:55:39.949454 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 754/1490] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:55:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:55:42.436218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12fe754-5684-4f12-bc9a-0b95aa3a8801 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:55:42.436778 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12fe754-5684-4f12-bc9a-0b95aa3a8801 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:55:42.436778 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12fe754-5684-4f12-bc9a-0b95aa3a8801 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:55:42.438712 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 737/1491] 10.222.0.17 () {54 vars in 927 bytes} [Sat Sep 10 23:55:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:55:42.444173 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a1f5d-467a-44fe-b9d8-5d4c3bcec88a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:55:42.444606 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a1f5d-467a-44fe-b9d8-5d4c3bcec88a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:55:42.444737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a1f5d-467a-44fe-b9d8-5d4c3bcec88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:55:42.513579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-472a1f5d-467a-44fe-b9d8-5d4c3bcec88a None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:55:42.514803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-472a1f5d-467a-44fe-b9d8-5d4c3bcec88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:55:42.611061 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 755/1492] 10.222.0.17 () {58 vars in 1024 bytes} [Sat Sep 10 23:55:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:55:42.911727 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12fe754-5684-4f12-bc9a-0b95aa3a8801 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:55:42.913311 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e12fe754-5684-4f12-bc9a-0b95aa3a8801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:55:43.018404 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12fe754-5684-4f12-bc9a-0b95aa3a8801 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:55:43.090930 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3432722b-10f3-4901-991c-988bca522124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:55:43.091532 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432722b-10f3-4901-991c-988bca522124 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:55:43.091810 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432722b-10f3-4901-991c-988bca522124 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:55:43.092082 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432722b-10f3-4901-991c-988bca522124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:55:43.093853 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3432722b-10f3-4901-991c-988bca522124 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:55:43.102605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:55:43.206938 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3432722b-10f3-4901-991c-988bca522124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:55:43.208380 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3432722b-10f3-4901-991c-988bca522124 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:55:43.226716 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 738/1493] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:55:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:55:52.609271 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=82 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:55:53.224475 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=50 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:57:28.506880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea3bd5-8492-4d4c-832a-c18b5ed8da0a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:57:28.507504 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea3bd5-8492-4d4c-832a-c18b5ed8da0a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:57:28.507504 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea3bd5-8492-4d4c-832a-c18b5ed8da0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:57:28.590990 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-e7ea3bd5-8492-4d4c-832a-c18b5ed8da0a None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:57:28.592133 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e7ea3bd5-8492-4d4c-832a-c18b5ed8da0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:57:28.695787 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 756/1494] 10.222.0.17 () {58 vars in 958 bytes} [Sat Sep 10 23:57:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:57:28.703961 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31279b3b-5cf2-4bb8-a2a3-5c4ea34042d0 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:57:28.704441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31279b3b-5cf2-4bb8-a2a3-5c4ea34042d0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:57:28.704546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31279b3b-5cf2-4bb8-a2a3-5c4ea34042d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:57:28.724161 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-31279b3b-5cf2-4bb8-a2a3-5c4ea34042d0 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:57:28.725355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-31279b3b-5cf2-4bb8-a2a3-5c4ea34042d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:57:28.733730 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 739/1495] 10.222.0.17 () {58 vars in 958 bytes} [Sat Sep 10 23:57:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:57:28.738776 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea3bd5-8492-4d4c-832a-c18b5ed8da0a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:57:28.748307 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea3bd5-8492-4d4c-832a-c18b5ed8da0a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:57:28.752956 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521b763-bdf3-414c-9356-d8c8dfd8794d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:57:28.753543 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521b763-bdf3-414c-9356-d8c8dfd8794d admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:57:28.753816 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521b763-bdf3-414c-9356-d8c8dfd8794d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:57:28.754081 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521b763-bdf3-414c-9356-d8c8dfd8794d admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:57:28.754737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521b763-bdf3-414c-9356-d8c8dfd8794d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 23:57:28.755035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521b763-bdf3-414c-9356-d8c8dfd8794d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:57:28.756024 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: WARNING py.warnings [None req-f521b763-bdf3-414c-9356-d8c8dfd8794d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 23:57:28.756024 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: warnings.warn(msg) Sep 10 23:57:28.756024 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: Sep 10 23:57:28.756896 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521b763-bdf3-414c-9356-d8c8dfd8794d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:57:28.768631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 757/1496] 10.222.0.17 () {58 vars in 1153 bytes} [Sat Sep 10 23:57:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:57:28.776704 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31279b3b-5cf2-4bb8-a2a3-5c4ea34042d0 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:57:28.785915 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31279b3b-5cf2-4bb8-a2a3-5c4ea34042d0 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:57:28.790508 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a95f4-90e7-44f6-a8c5-61ee7b18772d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:57:28.791123 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a95f4-90e7-44f6-a8c5-61ee7b18772d admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:57:28.791400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a95f4-90e7-44f6-a8c5-61ee7b18772d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:57:28.791667 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a95f4-90e7-44f6-a8c5-61ee7b18772d admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:57:28.792302 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a95f4-90e7-44f6-a8c5-61ee7b18772d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:57:28.794063 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: WARNING py.warnings [None req-699a95f4-90e7-44f6-a8c5-61ee7b18772d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 23:57:28.794063 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: warnings.warn(msg) Sep 10 23:57:28.794063 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: Sep 10 23:57:28.795293 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a95f4-90e7-44f6-a8c5-61ee7b18772d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:57:28.906369 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 740/1497] 10.222.0.17 () {60 vars in 1153 bytes} [Sat Sep 10 23:57:28 2022] POST /identity/v3/projects => generated 379 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 23:57:28.911323 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521b763-bdf3-414c-9356-d8c8dfd8794d admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:57:28.923614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521b763-bdf3-414c-9356-d8c8dfd8794d admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:57:28.979626 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:57:28.988277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cace3a3-19de-4b11-8459-2932cc71b30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:57:28.988893 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cace3a3-19de-4b11-8459-2932cc71b30b admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:57:28.992127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cace3a3-19de-4b11-8459-2932cc71b30b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:57:28.992127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cace3a3-19de-4b11-8459-2932cc71b30b admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:57:28.992127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cace3a3-19de-4b11-8459-2932cc71b30b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:57:28.992127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: WARNING py.warnings [None req-5cace3a3-19de-4b11-8459-2932cc71b30b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 23:57:28.992127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: warnings.warn(msg) Sep 10 23:57:28.992127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: Sep 10 23:57:28.992644 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cace3a3-19de-4b11-8459-2932cc71b30b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:57:29.079300 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 758/1498] 10.222.0.17 () {60 vars in 1144 bytes} [Sat Sep 10 23:57:28 2022] POST /identity/v3/users => generated 290 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 23:57:29.083659 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a95f4-90e7-44f6-a8c5-61ee7b18772d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:57:29.093900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a95f4-90e7-44f6-a8c5-61ee7b18772d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:57:29.098784 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af28a27-d746-47ca-b104-fe2ab366d406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:57:29.099374 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af28a27-d746-47ca-b104-fe2ab366d406 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:57:29.099651 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af28a27-d746-47ca-b104-fe2ab366d406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:57:29.099921 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af28a27-d746-47ca-b104-fe2ab366d406 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:57:29.100555 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af28a27-d746-47ca-b104-fe2ab366d406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:57:29.101569 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: WARNING py.warnings [None req-9af28a27-d746-47ca-b104-fe2ab366d406 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 23:57:29.101569 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: warnings.warn(msg) Sep 10 23:57:29.101569 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: Sep 10 23:57:29.102488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af28a27-d746-47ca-b104-fe2ab366d406 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:57:29.115509 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 741/1499] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Sep 10 23:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 23:57:29.119464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cace3a3-19de-4b11-8459-2932cc71b30b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:57:29.130166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cace3a3-19de-4b11-8459-2932cc71b30b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:57:29.134629 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88edba-596e-416c-ad6f-b2cadb0d127a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:57:29.135240 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88edba-596e-416c-ad6f-b2cadb0d127a admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:57:29.135515 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88edba-596e-416c-ad6f-b2cadb0d127a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:57:29.135777 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88edba-596e-416c-ad6f-b2cadb0d127a admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:57:29.136401 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88edba-596e-416c-ad6f-b2cadb0d127a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:57:29.137382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: WARNING py.warnings [None req-6d88edba-596e-416c-ad6f-b2cadb0d127a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 23:57:29.137382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: warnings.warn(msg) Sep 10 23:57:29.137382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: Sep 10 23:57:29.138251 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88edba-596e-416c-ad6f-b2cadb0d127a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:57:29.151940 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 759/1500] 10.222.0.17 () {58 vars in 1122 bytes} [Sat Sep 10 23:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 23:57:29.155942 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af28a27-d746-47ca-b104-fe2ab366d406 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:57:29.165589 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af28a27-d746-47ca-b104-fe2ab366d406 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:57:29.170090 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f4244-f0a2-487d-9124-4283aa741ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:57:29.170750 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f4244-f0a2-487d-9124-4283aa741ae6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:57:29.171031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f4244-f0a2-487d-9124-4283aa741ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:57:29.171313 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f4244-f0a2-487d-9124-4283aa741ae6 admin admin] PATH_INFO: `/v3/projects/cdf2654c9c3b48ff9e7f3b5e42e3366b/users/38e56c7772304eafbc4406642632c9a6/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:57:29.185622 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f4244-f0a2-487d-9124-4283aa741ae6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdf2654c9c3b48ff9e7f3b5e42e3366b, user_id=38e56c7772304eafbc4406642632c9a6, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:57:29.186639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: WARNING py.warnings [None req-a07f4244-f0a2-487d-9124-4283aa741ae6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 23:57:29.186639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: warnings.warn(msg) Sep 10 23:57:29.186639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: Sep 10 23:57:29.189125 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f4244-f0a2-487d-9124-4283aa741ae6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:57:29.252485 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 742/1501] 10.222.0.17 () {60 vars in 1483 bytes} [Sat Sep 10 23:57:29 2022] PUT /identity/v3/projects/cdf2654c9c3b48ff9e7f3b5e42e3366b/users/38e56c7772304eafbc4406642632c9a6/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:57:29.264509 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9c071-5db2-4f9b-abc8-94198edcd23b None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:57:29.264787 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9c071-5db2-4f9b-abc8-94198edcd23b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:57:29.265060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9c071-5db2-4f9b-abc8-94198edcd23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:57:29.293529 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-dcd9c071-5db2-4f9b-abc8-94198edcd23b None None] MFA Rules not processed for user `38e56c7772304eafbc4406642632c9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:57:29.294777 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-dcd9c071-5db2-4f9b-abc8-94198edcd23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:57:29.360173 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 760/1502] 10.222.0.17 () {58 vars in 958 bytes} [Sat Sep 10 23:57:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:57:29.373275 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aff6ed-20e9-4463-9b68-42dae1c65051 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:57:29.373556 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aff6ed-20e9-4463-9b68-42dae1c65051 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:57:29.373831 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aff6ed-20e9-4463-9b68-42dae1c65051 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:57:29.375221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 743/1503] 10.222.0.17 () {54 vars in 964 bytes} [Sat Sep 10 23:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:57:29.380238 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d6d2e-1ce5-4703-b493-3ce880cae56b None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:57:29.380518 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d6d2e-1ce5-4703-b493-3ce880cae56b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:57:29.380789 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d6d2e-1ce5-4703-b493-3ce880cae56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:57:29.439790 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a73d6d2e-1ce5-4703-b493-3ce880cae56b None None] MFA Rules not processed for user `c6fccf47e1ec4324983c5218ec3c96a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:57:29.440893 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a73d6d2e-1ce5-4703-b493-3ce880cae56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:57:29.514737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 761/1504] 10.222.0.17 () {58 vars in 1061 bytes} [Sat Sep 10 23:57:29 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:57:29.523159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aff6ed-20e9-4463-9b68-42dae1c65051 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:57:29.534436 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aff6ed-20e9-4463-9b68-42dae1c65051 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:57:29.537960 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169769da-4116-4eb9-8646-039b62d80830 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:57:29.538550 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169769da-4116-4eb9-8646-039b62d80830 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:57:29.543823 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169769da-4116-4eb9-8646-039b62d80830 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:57:29.543823 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169769da-4116-4eb9-8646-039b62d80830 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:57:29.547486 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169769da-4116-4eb9-8646-039b62d80830 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:57:29.548902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169769da-4116-4eb9-8646-039b62d80830 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:57:29.557488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 744/1505] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:57:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:57:29.603247 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9ee58-eff1-4f37-be7c-966bde296e0d None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:57:29.603535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9ee58-eff1-4f37-be7c-966bde296e0d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:57:29.603808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9ee58-eff1-4f37-be7c-966bde296e0d None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:57:29.605063 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 762/1506] 10.222.0.17 () {52 vars in 830 bytes} [Sat Sep 10 23:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:57:29.608586 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169769da-4116-4eb9-8646-039b62d80830 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:57:29.672640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169769da-4116-4eb9-8646-039b62d80830 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:57:29.677683 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a67f2-58b1-4cd9-a8af-8fa5da816f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:57:29.680101 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a67f2-58b1-4cd9-a8af-8fa5da816f3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:57:29.680101 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a67f2-58b1-4cd9-a8af-8fa5da816f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:57:29.680101 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a67f2-58b1-4cd9-a8af-8fa5da816f3b admin admin] PATH_INFO: `/v3/users/38e56c7772304eafbc4406642632c9a6` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:57:29.680101 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a67f2-58b1-4cd9-a8af-8fa5da816f3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=38e56c7772304eafbc4406642632c9a6)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:57:29.681102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: WARNING py.warnings [None req-db1a67f2-58b1-4cd9-a8af-8fa5da816f3b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 23:57:29.681102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: warnings.warn(msg) Sep 10 23:57:29.681102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: Sep 10 23:57:29.682314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a67f2-58b1-4cd9-a8af-8fa5da816f3b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:57:29.872196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-db1a67f2-58b1-4cd9-a8af-8fa5da816f3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38e56c7772304eafbc4406642632c9a6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 23:57:29.886689 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-db1a67f2-58b1-4cd9-a8af-8fa5da816f3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38e56c7772304eafbc4406642632c9a6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 23:57:29.910234 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-db1a67f2-58b1-4cd9-a8af-8fa5da816f3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38e56c7772304eafbc4406642632c9a6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 23:57:29.914298 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:57:29.977484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-db1a67f2-58b1-4cd9-a8af-8fa5da816f3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38e56c7772304eafbc4406642632c9a6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 23:57:29.978156 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-db1a67f2-58b1-4cd9-a8af-8fa5da816f3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38e56c7772304eafbc4406642632c9a6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 23:57:29.984934 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 745/1507] 10.222.0.17 () {58 vars in 1224 bytes} [Sat Sep 10 23:57:29 2022] DELETE /identity/v3/users/38e56c7772304eafbc4406642632c9a6 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:57:29.989298 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67702849-6019-4517-91cd-1c4f478aec4b None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:57:29.989576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67702849-6019-4517-91cd-1c4f478aec4b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:57:29.989847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67702849-6019-4517-91cd-1c4f478aec4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:57:30.008926 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-67702849-6019-4517-91cd-1c4f478aec4b None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:57:30.010131 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-67702849-6019-4517-91cd-1c4f478aec4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:57:30.019263 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:57:30.070006 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 763/1508] 10.222.0.17 () {58 vars in 958 bytes} [Sat Sep 10 23:57:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:57:30.081070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a67f2-58b1-4cd9-a8af-8fa5da816f3b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:57:30.082476 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-db1a67f2-58b1-4cd9-a8af-8fa5da816f3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:57:30.159055 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a67f2-58b1-4cd9-a8af-8fa5da816f3b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:57:30.162583 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7510c2b-ccd6-417d-81a1-f830bc0fd68a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:57:30.163211 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7510c2b-ccd6-417d-81a1-f830bc0fd68a service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:57:30.163490 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7510c2b-ccd6-417d-81a1-f830bc0fd68a service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:57:30.163762 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7510c2b-ccd6-417d-81a1-f830bc0fd68a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:57:30.173130 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7510c2b-ccd6-417d-81a1-f830bc0fd68a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:57:30.174593 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7510c2b-ccd6-417d-81a1-f830bc0fd68a service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:57:30.183879 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 746/1509] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:57:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:57:30.682349 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67702849-6019-4517-91cd-1c4f478aec4b None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:57:30.683926 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-67702849-6019-4517-91cd-1c4f478aec4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:57:30.696273 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67702849-6019-4517-91cd-1c4f478aec4b None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:57:30.701300 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e18c7f-4ec3-45c9-8aed-03b04a231a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:57:30.701940 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e18c7f-4ec3-45c9-8aed-03b04a231a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:57:30.702221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e18c7f-4ec3-45c9-8aed-03b04a231a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:57:30.702492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e18c7f-4ec3-45c9-8aed-03b04a231a9e admin admin] PATH_INFO: `/v3/projects/cdf2654c9c3b48ff9e7f3b5e42e3366b` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:57:30.703823 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e18c7f-4ec3-45c9-8aed-03b04a231a9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdf2654c9c3b48ff9e7f3b5e42e3366b)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:57:30.705022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: WARNING py.warnings [None req-32e18c7f-4ec3-45c9-8aed-03b04a231a9e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 23:57:30.705022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: warnings.warn(msg) Sep 10 23:57:30.705022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: Sep 10 23:57:30.706266 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e18c7f-4ec3-45c9-8aed-03b04a231a9e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:57:30.822343 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-32e18c7f-4ec3-45c9-8aed-03b04a231a9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdf2654c9c3b48ff9e7f3b5e42e3366b', 'request_id': 'req-32e18c7f-4ec3-45c9-8aed-03b04a231a9e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 23:57:30.914708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-32e18c7f-4ec3-45c9-8aed-03b04a231a9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdf2654c9c3b48ff9e7f3b5e42e3366b', 'request_id': 'req-32e18c7f-4ec3-45c9-8aed-03b04a231a9e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 23:57:30.921537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-32e18c7f-4ec3-45c9-8aed-03b04a231a9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdf2654c9c3b48ff9e7f3b5e42e3366b', 'request_id': 'req-32e18c7f-4ec3-45c9-8aed-03b04a231a9e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 23:57:30.940110 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-32e18c7f-4ec3-45c9-8aed-03b04a231a9e admin admin] The token cache is being invalidate because project cdf2654c9c3b48ff9e7f3b5e42e3366b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 23:57:30.940330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-32e18c7f-4ec3-45c9-8aed-03b04a231a9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 23:57:30.940961 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-32e18c7f-4ec3-45c9-8aed-03b04a231a9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 23:57:30.942868 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 764/1510] 10.222.0.17 () {58 vars in 1233 bytes} [Sat Sep 10 23:57:30 2022] DELETE /identity/v3/projects/cdf2654c9c3b48ff9e7f3b5e42e3366b => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 23:57:40.179742 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:57:40.936988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:58:57.918123 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7014f6-565d-4921-84b0-813d3a974d7f None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:58:57.918754 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7014f6-565d-4921-84b0-813d3a974d7f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:58:57.918754 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7014f6-565d-4921-84b0-813d3a974d7f None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:58:57.920022 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 747/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 23:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 23:58:57.925622 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6848b2-eedd-4cca-ae80-ef2edc130816 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:58:57.926079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6848b2-eedd-4cca-ae80-ef2edc130816 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:58:57.926186 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6848b2-eedd-4cca-ae80-ef2edc130816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:58:57.946166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-ca6848b2-eedd-4cca-ae80-ef2edc130816 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:58:57.947403 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ca6848b2-eedd-4cca-ae80-ef2edc130816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:58:58.006678 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 765/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 23:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:58:58.160602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7014f6-565d-4921-84b0-813d3a974d7f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:58:58.162353 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fb7014f6-565d-4921-84b0-813d3a974d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:58:58.263208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7014f6-565d-4921-84b0-813d3a974d7f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:58:58.268838 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f21dc-c6a5-4dc7-897f-272f01a335db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:58:58.269429 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f21dc-c6a5-4dc7-897f-272f01a335db service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:58:58.269703 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f21dc-c6a5-4dc7-897f-272f01a335db service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:58:58.269970 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f21dc-c6a5-4dc7-897f-272f01a335db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:58:58.277709 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f21dc-c6a5-4dc7-897f-272f01a335db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:58:58.279393 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f21dc-c6a5-4dc7-897f-272f01a335db service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:58:58.289031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 748/1513] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:58:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:58:59.373387 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbbad1-2799-45d9-a52a-2e88ccdda387 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:58:59.373894 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbbad1-2799-45d9-a52a-2e88ccdda387 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:58:59.374004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbbad1-2799-45d9-a52a-2e88ccdda387 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:58:59.375314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 766/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 23:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:58:59.382095 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b9bfd-e60f-423d-af1f-33517bf4e45e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:58:59.382533 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b9bfd-e60f-423d-af1f-33517bf4e45e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:58:59.382645 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b9bfd-e60f-423d-af1f-33517bf4e45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:58:59.402389 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-e47b9bfd-e60f-423d-af1f-33517bf4e45e None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:58:59.403627 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e47b9bfd-e60f-423d-af1f-33517bf4e45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:58:59.411731 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 749/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 23:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:58:59.467119 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92000205-4f55-4e2d-8d52-ec3e9cad3fb3 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:58:59.467479 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92000205-4f55-4e2d-8d52-ec3e9cad3fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:58:59.467838 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92000205-4f55-4e2d-8d52-ec3e9cad3fb3 None None] PATH_INFO: `/v3` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:58:59.469161 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 767/1516] 10.222.0.43 () {54 vars in 925 bytes} [Sat Sep 10 23:58:59 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 23:58:59.476016 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6d843-1dc2-458c-bff3-7a02e35b75c1 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:58:59.485794 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6d843-1dc2-458c-bff3-7a02e35b75c1 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:58:59.485794 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6d843-1dc2-458c-bff3-7a02e35b75c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:58:59.542054 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-aeb6d843-1dc2-458c-bff3-7a02e35b75c1 None None] MFA Rules not processed for user `5cb1d85a338e40f18e61887a1becb906`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:58:59.543309 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-aeb6d843-1dc2-458c-bff3-7a02e35b75c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:58:59.573401 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92000205-4f55-4e2d-8d52-ec3e9cad3fb3 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:58:59.589745 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92000205-4f55-4e2d-8d52-ec3e9cad3fb3 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:58:59.596145 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d578d-4ddd-4525-b3c3-6a46339f01c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:58:59.596744 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d578d-4ddd-4525-b3c3-6a46339f01c5 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:58:59.597024 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d578d-4ddd-4525-b3c3-6a46339f01c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:58:59.597294 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d578d-4ddd-4525-b3c3-6a46339f01c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:58:59.605592 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d578d-4ddd-4525-b3c3-6a46339f01c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:58:59.607271 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d578d-4ddd-4525-b3c3-6a46339f01c5 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:58:59.617235 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 750/1517] 10.222.0.43 () {58 vars in 1015 bytes} [Sat Sep 10 23:58:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:58:59.617418 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 768/1518] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:58:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:59:08.686507 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91923a-f5d7-48b3-8016-7456714d91a5 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:59:08.687067 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91923a-f5d7-48b3-8016-7456714d91a5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:59:08.687178 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91923a-f5d7-48b3-8016-7456714d91a5 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:59:08.689320 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 751/1519] 10.222.0.17 () {54 vars in 961 bytes} [Sat Sep 10 23:59:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:59:08.694617 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9bf19-d468-4e7d-ba85-39d8d025928b None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:59:08.695082 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9bf19-d468-4e7d-ba85-39d8d025928b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:59:08.695207 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9bf19-d468-4e7d-ba85-39d8d025928b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:59:08.714771 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-d2e9bf19-d468-4e7d-ba85-39d8d025928b None None] MFA Rules not processed for user `5cb1d85a338e40f18e61887a1becb906`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:59:08.715958 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-d2e9bf19-d468-4e7d-ba85-39d8d025928b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:59:08.724761 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 769/1520] 10.222.0.17 () {58 vars in 1058 bytes} [Sat Sep 10 23:59:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:59:08.732449 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b91923a-f5d7-48b3-8016-7456714d91a5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:59:08.745504 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b91923a-f5d7-48b3-8016-7456714d91a5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:59:08.750260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608cf59-01aa-4b2a-8e1b-bfbf2603fe76 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cb1d85a338e40f18e61887a1becb906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:59:08.751657 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608cf59-01aa-4b2a-8e1b-bfbf2603fe76 service placement] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:59:08.751933 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608cf59-01aa-4b2a-8e1b-bfbf2603fe76 service placement] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:59:08.752197 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608cf59-01aa-4b2a-8e1b-bfbf2603fe76 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:59:08.760180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5608cf59-01aa-4b2a-8e1b-bfbf2603fe76 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:59:08.761743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5608cf59-01aa-4b2a-8e1b-bfbf2603fe76 service placement] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:59:08.771628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 752/1521] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:59:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 23:59:11.103564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810de3e4-649c-46de-8e63-97d83ae3a962 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:59:11.104176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810de3e4-649c-46de-8e63-97d83ae3a962 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:59:11.104176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810de3e4-649c-46de-8e63-97d83ae3a962 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:59:11.105473 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 770/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 23:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:59:11.111971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2937a-b0b1-4057-85b5-76606fbbc253 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:59:11.112389 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2937a-b0b1-4057-85b5-76606fbbc253 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:59:11.112510 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2937a-b0b1-4057-85b5-76606fbbc253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:59:11.120127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:59:11.132791 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-82f2937a-b0b1-4057-85b5-76606fbbc253 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:59:11.146387 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-82f2937a-b0b1-4057-85b5-76606fbbc253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:59:11.146387 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 753/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 23:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 23:59:11.304615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810de3e4-649c-46de-8e63-97d83ae3a962 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:59:11.320216 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810de3e4-649c-46de-8e63-97d83ae3a962 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:59:11.325582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af847694-9a9b-47a0-84fb-1b6d9b066f83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:59:11.326164 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af847694-9a9b-47a0-84fb-1b6d9b066f83 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:59:11.326444 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af847694-9a9b-47a0-84fb-1b6d9b066f83 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:59:11.326733 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af847694-9a9b-47a0-84fb-1b6d9b066f83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:59:11.335175 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af847694-9a9b-47a0-84fb-1b6d9b066f83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:59:11.337972 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af847694-9a9b-47a0-84fb-1b6d9b066f83 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:59:11.345799 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 771/1524] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:59:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:59:12.430733 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d58f5de-85c8-4b19-b51d-8a17401a5d23 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:59:12.431329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d58f5de-85c8-4b19-b51d-8a17401a5d23 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:59:12.432669 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d58f5de-85c8-4b19-b51d-8a17401a5d23 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:59:12.434092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 754/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 23:59:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:59:12.441249 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222f700-4f25-4bd6-b2c6-6c820c9102e0 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:59:12.441858 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222f700-4f25-4bd6-b2c6-6c820c9102e0 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:59:12.441858 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222f700-4f25-4bd6-b2c6-6c820c9102e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:59:12.463777 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-9222f700-4f25-4bd6-b2c6-6c820c9102e0 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:59:12.464959 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9222f700-4f25-4bd6-b2c6-6c820c9102e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:59:12.472794 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 772/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 23:59:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:59:12.626463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d58f5de-85c8-4b19-b51d-8a17401a5d23 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:59:12.640269 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d58f5de-85c8-4b19-b51d-8a17401a5d23 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:59:12.646333 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfab560-6396-4d76-b4f5-306065a12242 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:59:12.646943 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfab560-6396-4d76-b4f5-306065a12242 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:59:12.647222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfab560-6396-4d76-b4f5-306065a12242 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:59:12.647489 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfab560-6396-4d76-b4f5-306065a12242 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:59:12.655613 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfab560-6396-4d76-b4f5-306065a12242 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:59:12.657443 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfab560-6396-4d76-b4f5-306065a12242 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:59:12.666566 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 755/1527] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:59:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:59:16.626422 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac365813-1dd6-45c3-bcc1-a8f2cc9eccd1 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:59:16.626990 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac365813-1dd6-45c3-bcc1-a8f2cc9eccd1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:59:16.626990 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac365813-1dd6-45c3-bcc1-a8f2cc9eccd1 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:59:16.628265 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 773/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 23:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:59:16.633613 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cffd2-e35b-40e7-9186-869813f94db0 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:59:16.634045 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cffd2-e35b-40e7-9186-869813f94db0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:59:16.634142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cffd2-e35b-40e7-9186-869813f94db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:59:16.653695 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-397cffd2-e35b-40e7-9186-869813f94db0 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:59:16.654905 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-397cffd2-e35b-40e7-9186-869813f94db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:59:16.663240 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 756/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 23:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:59:16.961772 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac365813-1dd6-45c3-bcc1-a8f2cc9eccd1 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:59:16.963492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ac365813-1dd6-45c3-bcc1-a8f2cc9eccd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:59:17.020311 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac365813-1dd6-45c3-bcc1-a8f2cc9eccd1 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:59:17.023844 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626ef04-da20-48d1-ade2-52fd8a864949 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:59:17.024447 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626ef04-da20-48d1-ade2-52fd8a864949 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:59:17.024723 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626ef04-da20-48d1-ade2-52fd8a864949 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:59:17.024994 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626ef04-da20-48d1-ade2-52fd8a864949 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:59:17.033971 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b626ef04-da20-48d1-ade2-52fd8a864949 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:59:17.035419 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b626ef04-da20-48d1-ade2-52fd8a864949 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:59:17.044468 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 774/1530] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:59:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:59:19.243120 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf78084-e27b-4391-9fc4-d482e3f95958 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:59:19.243676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf78084-e27b-4391-9fc4-d482e3f95958 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:59:19.243676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf78084-e27b-4391-9fc4-d482e3f95958 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:59:19.245921 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 757/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 23:59:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:59:19.251837 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab0d98-9eb1-4c0b-9bde-5f19ed202d3e None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:59:19.252312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab0d98-9eb1-4c0b-9bde-5f19ed202d3e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:59:19.252580 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab0d98-9eb1-4c0b-9bde-5f19ed202d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:59:19.276151 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-f1ab0d98-9eb1-4c0b-9bde-5f19ed202d3e None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:59:19.277343 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f1ab0d98-9eb1-4c0b-9bde-5f19ed202d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:59:19.285095 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 775/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 23:59:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:59:19.594780 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf78084-e27b-4391-9fc4-d482e3f95958 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:59:19.606860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf78084-e27b-4391-9fc4-d482e3f95958 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:59:19.611574 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6aff8-6a66-4cb2-9eb4-a028faa5bd7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:59:19.612316 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6aff8-6a66-4cb2-9eb4-a028faa5bd7b service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:59:19.612484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6aff8-6a66-4cb2-9eb4-a028faa5bd7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:59:19.612754 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6aff8-6a66-4cb2-9eb4-a028faa5bd7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:59:19.622564 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a6aff8-6a66-4cb2-9eb4-a028faa5bd7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:59:19.624204 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a6aff8-6a66-4cb2-9eb4-a028faa5bd7b service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:59:19.635595 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 758/1533] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:59:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:59:23.421374 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b94bd-9fc0-4537-85c4-5f3b0c53b3f4 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:59:23.422583 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b94bd-9fc0-4537-85c4-5f3b0c53b3f4 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:59:23.422583 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b94bd-9fc0-4537-85c4-5f3b0c53b3f4 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:59:23.423805 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 776/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 23:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:59:23.428968 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdddc6ac-a20b-4eee-ad73-55ccdabbf985 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:59:23.429380 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdddc6ac-a20b-4eee-ad73-55ccdabbf985 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:59:23.429513 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdddc6ac-a20b-4eee-ad73-55ccdabbf985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:59:23.448764 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-cdddc6ac-a20b-4eee-ad73-55ccdabbf985 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:59:23.449954 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-cdddc6ac-a20b-4eee-ad73-55ccdabbf985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:59:23.458148 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 759/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 23:59:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:59:23.774808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1b94bd-9fc0-4537-85c4-5f3b0c53b3f4 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:59:23.776472 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0b1b94bd-9fc0-4537-85c4-5f3b0c53b3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:59:23.782767 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=75 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:59:23.834205 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1b94bd-9fc0-4537-85c4-5f3b0c53b3f4 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:59:23.837687 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6044ce3f-cf9d-4a00-90f4-7775fda13e16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:59:23.838281 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044ce3f-cf9d-4a00-90f4-7775fda13e16 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:59:23.838553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044ce3f-cf9d-4a00-90f4-7775fda13e16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:59:23.838852 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044ce3f-cf9d-4a00-90f4-7775fda13e16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:59:23.848099 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6044ce3f-cf9d-4a00-90f4-7775fda13e16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:59:23.849506 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6044ce3f-cf9d-4a00-90f4-7775fda13e16 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:59:23.858521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 777/1536] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:59:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:59:25.999603 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2d2db-f391-471e-ac08-c9fcd35c6d99 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:59:26.000700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2d2db-f391-471e-ac08-c9fcd35c6d99 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:59:26.000700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2d2db-f391-471e-ac08-c9fcd35c6d99 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:59:26.002128 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 760/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 23:59:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 23:59:26.007487 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0804bf-4284-4a1f-a1b6-fe90c13b8b4e None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:59:26.007938 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0804bf-4284-4a1f-a1b6-fe90c13b8b4e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:59:26.008045 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0804bf-4284-4a1f-a1b6-fe90c13b8b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:59:26.027679 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-9f0804bf-4284-4a1f-a1b6-fe90c13b8b4e None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 23:59:26.028853 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9f0804bf-4284-4a1f-a1b6-fe90c13b8b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:59:26.036382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 778/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 23:59:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 23:59:26.346317 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2d2db-f391-471e-ac08-c9fcd35c6d99 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 23:59:26.348016 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2fc2d2db-f391-471e-ac08-c9fcd35c6d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 23:59:26.360856 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2d2db-f391-471e-ac08-c9fcd35c6d99 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 23:59:26.364329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a40015-0ab8-4058-b54a-a8b5763d0380 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 23:59:26.364917 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a40015-0ab8-4058-b54a-a8b5763d0380 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 23:59:26.365194 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a40015-0ab8-4058-b54a-a8b5763d0380 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 23:59:26.365465 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a40015-0ab8-4058-b54a-a8b5763d0380 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 23:59:26.374213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a40015-0ab8-4058-b54a-a8b5763d0380 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 23:59:26.375855 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a40015-0ab8-4058-b54a-a8b5763d0380 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 23:59:26.385035 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 761/1539] 10.222.0.17 () {60 vars in 1365 bytes} [Sat Sep 10 23:59:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 23:59:36.022311 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=28 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 23:59:36.380938 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=26 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:00:51.102274 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f692806-4b98-4576-87ac-c7d17af6814d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.102949 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f692806-4b98-4576-87ac-c7d17af6814d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.102949 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f692806-4b98-4576-87ac-c7d17af6814d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.135768 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-3f692806-4b98-4576-87ac-c7d17af6814d None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:51.137005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3f692806-4b98-4576-87ac-c7d17af6814d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:51.152743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 779/1540] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:00:51.161788 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e6488-60cd-40f2-a316-99cd0783ac62 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.162384 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e6488-60cd-40f2-a316-99cd0783ac62 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.162384 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e6488-60cd-40f2-a316-99cd0783ac62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.195901 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d84e6488-60cd-40f2-a316-99cd0783ac62 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:51.197136 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d84e6488-60cd-40f2-a316-99cd0783ac62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:51.206565 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 762/1541] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:00:51.211419 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f692806-4b98-4576-87ac-c7d17af6814d None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:51.224970 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f692806-4b98-4576-87ac-c7d17af6814d None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:51.233321 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5486afd-59c7-41aa-a232-52f614d65629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:51.233936 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5486afd-59c7-41aa-a232-52f614d65629 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.234212 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5486afd-59c7-41aa-a232-52f614d65629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.234484 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5486afd-59c7-41aa-a232-52f614d65629 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.235236 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5486afd-59c7-41aa-a232-52f614d65629 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:00:51.235577 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5486afd-59c7-41aa-a232-52f614d65629 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:51.237622 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5486afd-59c7-41aa-a232-52f614d65629 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:51.250358 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 780/1542] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:00:51.259374 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e6488-60cd-40f2-a316-99cd0783ac62 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:51.269599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e6488-60cd-40f2-a316-99cd0783ac62 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:51.279126 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88f1c0-fec9-434b-aa2e-d9d3f3564abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:51.279734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88f1c0-fec9-434b-aa2e-d9d3f3564abb admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.280011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88f1c0-fec9-434b-aa2e-d9d3f3564abb admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.280282 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88f1c0-fec9-434b-aa2e-d9d3f3564abb admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.280927 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88f1c0-fec9-434b-aa2e-d9d3f3564abb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:51.283160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88f1c0-fec9-434b-aa2e-d9d3f3564abb admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:51.323803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd1dc7-68d6-472a-8463-50227f0bb6a8 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.324175 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd1dc7-68d6-472a-8463-50227f0bb6a8 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.324356 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd1dc7-68d6-472a-8463-50227f0bb6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.351746 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-c6dd1dc7-68d6-472a-8463-50227f0bb6a8 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:51.353017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c6dd1dc7-68d6-472a-8463-50227f0bb6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:51.361513 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 781/1543] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:00:51.365029 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397af75-d5f1-4729-984e-b38681c955d7 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.365308 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397af75-d5f1-4729-984e-b38681c955d7 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.365585 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397af75-d5f1-4729-984e-b38681c955d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.381757 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 763/1544] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:00:51 2022] POST /identity/v3/projects => generated 363 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:00:51.383043 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1174b7c-07f0-4f15-be8c-f64de650b1c9 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.383324 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1174b7c-07f0-4f15-be8c-f64de650b1c9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.383594 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1174b7c-07f0-4f15-be8c-f64de650b1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.414751 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-6397af75-d5f1-4729-984e-b38681c955d7 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:51.415974 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6397af75-d5f1-4729-984e-b38681c955d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:51.422697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f1174b7c-07f0-4f15-be8c-f64de650b1c9 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:51.427340 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f1174b7c-07f0-4f15-be8c-f64de650b1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:51.485953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 782/1545] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:00:51.487268 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c643e-ba88-4638-913a-7a89c6aecbda None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.487551 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c643e-ba88-4638-913a-7a89c6aecbda None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.487822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c643e-ba88-4638-913a-7a89c6aecbda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.495121 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 764/1546] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:00:51.496392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b8af09-786f-41d8-ad8d-59be9b1be048 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.496680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b8af09-786f-41d8-ad8d-59be9b1be048 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.496957 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b8af09-786f-41d8-ad8d-59be9b1be048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.510466 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-173c643e-ba88-4638-913a-7a89c6aecbda None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:51.511713 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-173c643e-ba88-4638-913a-7a89c6aecbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:51.518631 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-58b8af09-786f-41d8-ad8d-59be9b1be048 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:51.519819 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-58b8af09-786f-41d8-ad8d-59be9b1be048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:51.520489 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 783/1547] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:00:51.524215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c643e-ba88-4638-913a-7a89c6aecbda None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:51.530586 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c643e-ba88-4638-913a-7a89c6aecbda None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:51.531647 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 765/1548] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:00:51.533065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19540d21-e558-49f1-8aaf-5873d2737e3d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.533419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19540d21-e558-49f1-8aaf-5873d2737e3d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.533689 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19540d21-e558-49f1-8aaf-5873d2737e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.541808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7f162-2de9-4002-a7a1-956e35fce1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:51.542476 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7f162-2de9-4002-a7a1-956e35fce1ec admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.542789 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7f162-2de9-4002-a7a1-956e35fce1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.543056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7f162-2de9-4002-a7a1-956e35fce1ec admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.543730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc7f162-2de9-4002-a7a1-956e35fce1ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:51.545647 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc7f162-2de9-4002-a7a1-956e35fce1ec admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:51.556155 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-19540d21-e558-49f1-8aaf-5873d2737e3d None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:51.557413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-19540d21-e558-49f1-8aaf-5873d2737e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:51.570989 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 766/1549] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:00:51.572267 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1763c7-0b2f-45ea-80fd-a42b76bf3d1f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.572551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1763c7-0b2f-45ea-80fd-a42b76bf3d1f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.572825 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1763c7-0b2f-45ea-80fd-a42b76bf3d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.592529 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-6d1763c7-0b2f-45ea-80fd-a42b76bf3d1f None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:51.593755 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6d1763c7-0b2f-45ea-80fd-a42b76bf3d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:51.602168 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 767/1550] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:00:51.606191 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0bf43c-201c-48ed-951e-c9979767c142 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.606191 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0bf43c-201c-48ed-951e-c9979767c142 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.606191 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0bf43c-201c-48ed-951e-c9979767c142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.623949 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5b0bf43c-201c-48ed-951e-c9979767c142 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:51.626861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5b0bf43c-201c-48ed-951e-c9979767c142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:51.630697 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 784/1551] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:00:51 2022] POST /identity/v3/users => generated 282 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:00:51.631634 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df22f7-4112-4b7d-a438-54c220ed2188 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.631915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df22f7-4112-4b7d-a438-54c220ed2188 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.632188 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df22f7-4112-4b7d-a438-54c220ed2188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.638466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 768/1552] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:00:51.639772 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eeff71-b4e5-4fa1-b65b-60161df6fc95 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.640054 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eeff71-b4e5-4fa1-b65b-60161df6fc95 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.646956 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eeff71-b4e5-4fa1-b65b-60161df6fc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.652004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-10df22f7-4112-4b7d-a438-54c220ed2188 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:51.655090 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-10df22f7-4112-4b7d-a438-54c220ed2188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:51.662835 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 785/1553] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:00:51.663444 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df22f7-4112-4b7d-a438-54c220ed2188 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:51.667398 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-89eeff71-b4e5-4fa1-b65b-60161df6fc95 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:51.668620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-89eeff71-b4e5-4fa1-b65b-60161df6fc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:51.672680 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df22f7-4112-4b7d-a438-54c220ed2188 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:51.677454 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 769/1554] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:00:51.678241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c7216-fb52-492d-a862-6f1ca5ce7142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:51.678859 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c7216-fb52-492d-a862-6f1ca5ce7142 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.678990 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a188c-5e49-413c-a3fc-f496c34e2b70 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.679160 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c7216-fb52-492d-a862-6f1ca5ce7142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.679271 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a188c-5e49-413c-a3fc-f496c34e2b70 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.679446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c7216-fb52-492d-a862-6f1ca5ce7142 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.679544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a188c-5e49-413c-a3fc-f496c34e2b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.680250 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c7216-fb52-492d-a862-6f1ca5ce7142 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:00:51.680555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c7216-fb52-492d-a862-6f1ca5ce7142 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:51.682143 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c7216-fb52-492d-a862-6f1ca5ce7142 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:51.695241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 786/1555] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:00:51.696298 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a0ccb-4fc0-4d26-a87f-ebd36270da2a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.696580 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a0ccb-4fc0-4d26-a87f-ebd36270da2a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.696848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a0ccb-4fc0-4d26-a87f-ebd36270da2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.699710 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0c7a188c-5e49-413c-a3fc-f496c34e2b70 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:51.700918 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0c7a188c-5e49-413c-a3fc-f496c34e2b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:51.708967 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 770/1556] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:00:51.712491 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4a7a9-5a6e-4a32-b38c-5faa233f4b63 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.712782 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4a7a9-5a6e-4a32-b38c-5faa233f4b63 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.713054 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4a7a9-5a6e-4a32-b38c-5faa233f4b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.717015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b35a0ccb-4fc0-4d26-a87f-ebd36270da2a None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:51.718184 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b35a0ccb-4fc0-4d26-a87f-ebd36270da2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:51.726436 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 787/1557] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:00:51.727436 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a0ccb-4fc0-4d26-a87f-ebd36270da2a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:51.736626 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c1b4a7a9-5a6e-4a32-b38c-5faa233f4b63 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:51.738058 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c1b4a7a9-5a6e-4a32-b38c-5faa233f4b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:51.741693 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a0ccb-4fc0-4d26-a87f-ebd36270da2a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:51.746220 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 771/1558] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:00:51.746344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa99cf5-afe8-4c6b-8e60-510fc7970e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:51.747427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c4842-5aa1-4dbb-bd04-3ec9e442a57a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.747706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c4842-5aa1-4dbb-bd04-3ec9e442a57a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.747975 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c4842-5aa1-4dbb-bd04-3ec9e442a57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.750492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa99cf5-afe8-4c6b-8e60-510fc7970e7d admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.750810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa99cf5-afe8-4c6b-8e60-510fc7970e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.751160 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa99cf5-afe8-4c6b-8e60-510fc7970e7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.752742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa99cf5-afe8-4c6b-8e60-510fc7970e7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:51.754363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa99cf5-afe8-4c6b-8e60-510fc7970e7d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:51.765023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 788/1559] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:00:51.766023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa99cf5-afe8-4c6b-8e60-510fc7970e7d admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:51.767575 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-6a5c4842-5aa1-4dbb-bd04-3ec9e442a57a None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:51.768757 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6a5c4842-5aa1-4dbb-bd04-3ec9e442a57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:51.775513 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa99cf5-afe8-4c6b-8e60-510fc7970e7d admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:51.776459 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 772/1560] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:00:51.777237 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5c4842-5aa1-4dbb-bd04-3ec9e442a57a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:51.780610 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96e887-1c7d-4372-8305-627d240dd457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:51.781205 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96e887-1c7d-4372-8305-627d240dd457 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.781518 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96e887-1c7d-4372-8305-627d240dd457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.781791 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96e887-1c7d-4372-8305-627d240dd457 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.782597 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96e887-1c7d-4372-8305-627d240dd457 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:00:51.783267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96e887-1c7d-4372-8305-627d240dd457 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:51.784945 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96e887-1c7d-4372-8305-627d240dd457 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:51.786237 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5c4842-5aa1-4dbb-bd04-3ec9e442a57a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:51.790739 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b7401-c3fe-4ca8-9f70-6d3508ef8ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:51.791312 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b7401-c3fe-4ca8-9f70-6d3508ef8ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.791973 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b7401-c3fe-4ca8-9f70-6d3508ef8ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.791973 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b7401-c3fe-4ca8-9f70-6d3508ef8ef4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.792458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b7401-c3fe-4ca8-9f70-6d3508ef8ef4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:00:51.792750 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b7401-c3fe-4ca8-9f70-6d3508ef8ef4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:51.793740 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: WARNING py.warnings [None req-4b2b7401-c3fe-4ca8-9f70-6d3508ef8ef4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 11 00:00:51.793740 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: warnings.warn(msg) Sep 11 00:00:51.793740 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: Sep 11 00:00:51.794604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b7401-c3fe-4ca8-9f70-6d3508ef8ef4 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:51.798497 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 789/1561] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:00:51.799712 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96e887-1c7d-4372-8305-627d240dd457 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:51.809287 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96e887-1c7d-4372-8305-627d240dd457 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:51.811289 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 773/1562] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:00:51.812347 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b7401-c3fe-4ca8-9f70-6d3508ef8ef4 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:51.814813 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ebc7d2-d115-4d02-9fc2-6511e574c184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:51.815479 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ebc7d2-d115-4d02-9fc2-6511e574c184 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.815752 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ebc7d2-d115-4d02-9fc2-6511e574c184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.816021 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ebc7d2-d115-4d02-9fc2-6511e574c184 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.816648 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ebc7d2-d115-4d02-9fc2-6511e574c184 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:51.818105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: WARNING py.warnings [None req-27ebc7d2-d115-4d02-9fc2-6511e574c184 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 00:00:51.818105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: warnings.warn(msg) Sep 11 00:00:51.818105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: Sep 11 00:00:51.820286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ebc7d2-d115-4d02-9fc2-6511e574c184 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:51.822865 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b7401-c3fe-4ca8-9f70-6d3508ef8ef4 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:51.827448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78238f-43f8-4d90-b41c-98b048b0abb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:51.828029 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78238f-43f8-4d90-b41c-98b048b0abb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.828296 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78238f-43f8-4d90-b41c-98b048b0abb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.828560 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78238f-43f8-4d90-b41c-98b048b0abb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.829277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78238f-43f8-4d90-b41c-98b048b0abb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:00:51.829579 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78238f-43f8-4d90-b41c-98b048b0abb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:51.831183 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78238f-43f8-4d90-b41c-98b048b0abb0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:51.842910 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 774/1563] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:00:51.843701 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78238f-43f8-4d90-b41c-98b048b0abb0 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:51.854640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78238f-43f8-4d90-b41c-98b048b0abb0 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:51.860872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590487ad-e05c-4f3c-9fcf-c0735739a8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:51.860872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590487ad-e05c-4f3c-9fcf-c0735739a8d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.860872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590487ad-e05c-4f3c-9fcf-c0735739a8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.860872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590487ad-e05c-4f3c-9fcf-c0735739a8d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.861600 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590487ad-e05c-4f3c-9fcf-c0735739a8d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:00:51.861899 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590487ad-e05c-4f3c-9fcf-c0735739a8d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:51.863512 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590487ad-e05c-4f3c-9fcf-c0735739a8d8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:51.875334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 775/1564] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:00:51.876840 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590487ad-e05c-4f3c-9fcf-c0735739a8d8 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:51.886639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590487ad-e05c-4f3c-9fcf-c0735739a8d8 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:51.891237 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a46bb63-ef1d-4cd5-99a2-2957482c8e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:51.891846 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46bb63-ef1d-4cd5-99a2-2957482c8e2b admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.892381 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46bb63-ef1d-4cd5-99a2-2957482c8e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.892722 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46bb63-ef1d-4cd5-99a2-2957482c8e2b admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.893351 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a46bb63-ef1d-4cd5-99a2-2957482c8e2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:00:51.893652 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a46bb63-ef1d-4cd5-99a2-2957482c8e2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:51.895258 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a46bb63-ef1d-4cd5-99a2-2957482c8e2b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:51.907052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 776/1565] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:00:51.908906 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a46bb63-ef1d-4cd5-99a2-2957482c8e2b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:51.915037 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 790/1566] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:00:51 2022] POST /identity/v3/projects => generated 393 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:00:51.915037 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ebc7d2-d115-4d02-9fc2-6511e574c184 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:51.919317 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a46bb63-ef1d-4cd5-99a2-2957482c8e2b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:51.944424 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:00:51.958121 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:00:51.976812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce676322-0a9e-4a43-afbd-d2aac6b707f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:51.977449 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce676322-0a9e-4a43-afbd-d2aac6b707f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.977728 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce676322-0a9e-4a43-afbd-d2aac6b707f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.977999 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce676322-0a9e-4a43-afbd-d2aac6b707f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.978635 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce676322-0a9e-4a43-afbd-d2aac6b707f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:51.979278 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ebc7d2-d115-4d02-9fc2-6511e574c184 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:51.980315 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce676322-0a9e-4a43-afbd-d2aac6b707f2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:51.984186 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f6d90-d3ba-4d98-a45f-85d1520053ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:51.984782 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f6d90-d3ba-4d98-a45f-85d1520053ef admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:51.985055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f6d90-d3ba-4d98-a45f-85d1520053ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:51.985324 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f6d90-d3ba-4d98-a45f-85d1520053ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:51.986159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f6d90-d3ba-4d98-a45f-85d1520053ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:00:51.986497 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f6d90-d3ba-4d98-a45f-85d1520053ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:51.988267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f6d90-d3ba-4d98-a45f-85d1520053ef admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:51.993359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 777/1567] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:00:51.993359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce676322-0a9e-4a43-afbd-d2aac6b707f2 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:52.000711 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce676322-0a9e-4a43-afbd-d2aac6b707f2 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:52.000865 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 791/1568] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:00:52.002288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f6d90-d3ba-4d98-a45f-85d1520053ef admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:52.006652 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57368ee2-8825-40fe-a090-b322a89720bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:52.009553 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57368ee2-8825-40fe-a090-b322a89720bf admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:52.009553 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57368ee2-8825-40fe-a090-b322a89720bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:52.009553 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57368ee2-8825-40fe-a090-b322a89720bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:52.009553 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57368ee2-8825-40fe-a090-b322a89720bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:52.010345 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57368ee2-8825-40fe-a090-b322a89720bf admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:52.012609 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f6d90-d3ba-4d98-a45f-85d1520053ef admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:52.017750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef4a5c-c2af-4a9b-bf2b-6c39eb9c899b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:52.018373 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef4a5c-c2af-4a9b-bf2b-6c39eb9c899b admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:52.018644 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef4a5c-c2af-4a9b-bf2b-6c39eb9c899b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:52.018939 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef4a5c-c2af-4a9b-bf2b-6c39eb9c899b admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:52.019556 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef4a5c-c2af-4a9b-bf2b-6c39eb9c899b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:52.021506 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef4a5c-c2af-4a9b-bf2b-6c39eb9c899b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:52.092753 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 778/1569] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:00:51 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:00:52.093558 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57368ee2-8825-40fe-a090-b322a89720bf admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:52.145513 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 792/1570] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:00:52 2022] POST /identity/v3/projects => generated 379 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:00:52.147125 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef4a5c-c2af-4a9b-bf2b-6c39eb9c899b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:52.155439 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57368ee2-8825-40fe-a090-b322a89720bf admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:52.209166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef4a5c-c2af-4a9b-bf2b-6c39eb9c899b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:52.212272 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec86b41-0e80-4c4e-b4bb-25c59efa0e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:52.212878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec86b41-0e80-4c4e-b4bb-25c59efa0e63 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:52.213156 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec86b41-0e80-4c4e-b4bb-25c59efa0e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:52.213424 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec86b41-0e80-4c4e-b4bb-25c59efa0e63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:52.213704 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3704a6-bee4-4779-8ace-a4a137e512cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:52.214044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec86b41-0e80-4c4e-b4bb-25c59efa0e63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:52.214299 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3704a6-bee4-4779-8ace-a4a137e512cd admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:52.214576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3704a6-bee4-4779-8ace-a4a137e512cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:52.214874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3704a6-bee4-4779-8ace-a4a137e512cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:52.215499 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3704a6-bee4-4779-8ace-a4a137e512cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:52.216095 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec86b41-0e80-4c4e-b4bb-25c59efa0e63 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:52.217397 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3704a6-bee4-4779-8ace-a4a137e512cd admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:52.315761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 779/1571] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:00:52 2022] POST /identity/v3/projects => generated 373 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:00:52.316574 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec86b41-0e80-4c4e-b4bb-25c59efa0e63 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:52.327615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 793/1572] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:00:52 2022] POST /identity/v3/projects => generated 389 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:00:52.328633 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3704a6-bee4-4779-8ace-a4a137e512cd admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:52.355854 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:00:52.383566 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec86b41-0e80-4c4e-b4bb-25c59efa0e63 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:52.393335 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3704a6-bee4-4779-8ace-a4a137e512cd admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:52.398521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af053b0-f8df-4724-9f48-193e564780da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:52.399134 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af053b0-f8df-4724-9f48-193e564780da admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:52.399413 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af053b0-f8df-4724-9f48-193e564780da admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:52.399683 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af053b0-f8df-4724-9f48-193e564780da admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:52.400316 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af053b0-f8df-4724-9f48-193e564780da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:52.402227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af053b0-f8df-4724-9f48-193e564780da admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:52.441245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41fcb5-7c86-4b25-8581-b1ce26d6db6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:52.441849 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41fcb5-7c86-4b25-8581-b1ce26d6db6a admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:52.442125 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41fcb5-7c86-4b25-8581-b1ce26d6db6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:52.442395 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41fcb5-7c86-4b25-8581-b1ce26d6db6a admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:52.443323 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41fcb5-7c86-4b25-8581-b1ce26d6db6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:52.444328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: WARNING py.warnings [None req-1d41fcb5-7c86-4b25-8581-b1ce26d6db6a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 00:00:52.444328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: warnings.warn(msg) Sep 11 00:00:52.444328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: Sep 11 00:00:52.445558 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41fcb5-7c86-4b25-8581-b1ce26d6db6a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:52.541733 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 794/1573] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:00:52 2022] POST /identity/v3/projects => generated 375 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:00:52.541733 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af053b0-f8df-4724-9f48-193e564780da admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:52.562221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 780/1574] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:00:52 2022] POST /identity/v3/users => generated 297 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:00:52.562221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41fcb5-7c86-4b25-8581-b1ce26d6db6a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:52.564272 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:00:52.572745 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41fcb5-7c86-4b25-8581-b1ce26d6db6a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:52.608199 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af053b0-f8df-4724-9f48-193e564780da admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:52.614161 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564fb73-f615-4c94-a644-9350de8ff816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:52.615011 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564fb73-f615-4c94-a644-9350de8ff816 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:52.615303 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564fb73-f615-4c94-a644-9350de8ff816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:52.615569 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564fb73-f615-4c94-a644-9350de8ff816 admin admin] PATH_INFO: `/v3/projects/0ec8e2b8bb6b4cfbbe9dc3b996700e5f/users/454283506bad4c978d1da1fef5813c4a/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:52.629524 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228491dc-9cb3-432c-90c7-218d12d5adba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:52.630147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228491dc-9cb3-432c-90c7-218d12d5adba admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:52.630278 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9564fb73-f615-4c94-a644-9350de8ff816 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ec8e2b8bb6b4cfbbe9dc3b996700e5f, user_id=454283506bad4c978d1da1fef5813c4a, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:52.630413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228491dc-9cb3-432c-90c7-218d12d5adba admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:52.630696 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228491dc-9cb3-432c-90c7-218d12d5adba admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:52.631192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: WARNING py.warnings [None req-9564fb73-f615-4c94-a644-9350de8ff816 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 00:00:52.631192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: warnings.warn(msg) Sep 11 00:00:52.631192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: Sep 11 00:00:52.631453 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228491dc-9cb3-432c-90c7-218d12d5adba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:52.633202 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228491dc-9cb3-432c-90c7-218d12d5adba admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:52.639305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9564fb73-f615-4c94-a644-9350de8ff816 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:52.720704 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 795/1575] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:00:52 2022] PUT /identity/v3/projects/0ec8e2b8bb6b4cfbbe9dc3b996700e5f/users/454283506bad4c978d1da1fef5813c4a/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:00:52.721587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564fb73-f615-4c94-a644-9350de8ff816 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:52.745799 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 781/1576] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:00:52 2022] POST /identity/v3/projects => generated 373 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:00:52.747011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228491dc-9cb3-432c-90c7-218d12d5adba admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:52.787526 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564fb73-f615-4c94-a644-9350de8ff816 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:52.806493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:00:52.812156 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228491dc-9cb3-432c-90c7-218d12d5adba admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:52.817427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8154db8-60ce-4962-9c46-0a70fb580b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:52.818289 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8154db8-60ce-4962-9c46-0a70fb580b65 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:52.818574 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8154db8-60ce-4962-9c46-0a70fb580b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:52.818879 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8154db8-60ce-4962-9c46-0a70fb580b65 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:52.819515 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8154db8-60ce-4962-9c46-0a70fb580b65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:52.821455 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8154db8-60ce-4962-9c46-0a70fb580b65 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:52.845810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7278614-2e90-4c9a-b854-a7f44094c7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:52.846414 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7278614-2e90-4c9a-b854-a7f44094c7a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:52.846746 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7278614-2e90-4c9a-b854-a7f44094c7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:52.847020 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7278614-2e90-4c9a-b854-a7f44094c7a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:52.847641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7278614-2e90-4c9a-b854-a7f44094c7a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:52.849537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7278614-2e90-4c9a-b854-a7f44094c7a9 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:52.892849 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 782/1577] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:00:52 2022] POST /identity/v3/users => generated 290 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:00:52.893753 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8154db8-60ce-4962-9c46-0a70fb580b65 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:52.905185 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8154db8-60ce-4962-9c46-0a70fb580b65 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:52.910732 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202153c0-6332-4de0-b28b-811d3a1be977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:52.911328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202153c0-6332-4de0-b28b-811d3a1be977 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:52.911605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202153c0-6332-4de0-b28b-811d3a1be977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:52.911876 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202153c0-6332-4de0-b28b-811d3a1be977 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:52.912502 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202153c0-6332-4de0-b28b-811d3a1be977 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:52.914417 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202153c0-6332-4de0-b28b-811d3a1be977 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:52.915628 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 796/1578] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:00:52 2022] POST /identity/v3/users => generated 292 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:00:52.917370 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7278614-2e90-4c9a-b854-a7f44094c7a9 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:52.927500 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7278614-2e90-4c9a-b854-a7f44094c7a9 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:52.932288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f031fc4-ef3a-4d1c-beae-74f2eeca9d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:52.932888 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f031fc4-ef3a-4d1c-beae-74f2eeca9d90 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:52.933195 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f031fc4-ef3a-4d1c-beae-74f2eeca9d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:52.933467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f031fc4-ef3a-4d1c-beae-74f2eeca9d90 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:52.934110 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f031fc4-ef3a-4d1c-beae-74f2eeca9d90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:52.936124 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f031fc4-ef3a-4d1c-beae-74f2eeca9d90 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:52.983773 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 783/1579] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:00:52 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:00:52.984543 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202153c0-6332-4de0-b28b-811d3a1be977 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:52.999353 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202153c0-6332-4de0-b28b-811d3a1be977 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:53.005380 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2763b-a2bf-4bf5-bb64-c21aa41d97c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:53.005979 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2763b-a2bf-4bf5-bb64-c21aa41d97c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:53.006257 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2763b-a2bf-4bf5-bb64-c21aa41d97c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:53.006529 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2763b-a2bf-4bf5-bb64-c21aa41d97c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:53.007193 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e2763b-a2bf-4bf5-bb64-c21aa41d97c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:53.009123 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e2763b-a2bf-4bf5-bb64-c21aa41d97c7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:53.035371 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 797/1580] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:00:52 2022] POST /identity/v3/users => generated 295 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:00:53.036351 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f031fc4-ef3a-4d1c-beae-74f2eeca9d90 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:53.046012 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f031fc4-ef3a-4d1c-beae-74f2eeca9d90 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:53.051236 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980480a-9194-49e9-990f-ae54105de4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:53.051841 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980480a-9194-49e9-990f-ae54105de4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:53.052137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980480a-9194-49e9-990f-ae54105de4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:53.052403 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980480a-9194-49e9-990f-ae54105de4e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:53.053019 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980480a-9194-49e9-990f-ae54105de4e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:53.054625 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980480a-9194-49e9-990f-ae54105de4e7 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:53.065854 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 798/1581] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:00:53.067201 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4955b-7f1d-4a9b-9d6e-ffe2ed785032 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:53.068143 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4955b-7f1d-4a9b-9d6e-ffe2ed785032 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:53.068143 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4955b-7f1d-4a9b-9d6e-ffe2ed785032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:53.087879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:00:53.095467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-dfe4955b-7f1d-4a9b-9d6e-ffe2ed785032 None None] MFA Rules not processed for user `454283506bad4c978d1da1fef5813c4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:53.096696 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-dfe4955b-7f1d-4a9b-9d6e-ffe2ed785032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:53.106729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 784/1582] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:00:52 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:00:53.106729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2763b-a2bf-4bf5-bb64-c21aa41d97c7 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:53.116982 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2763b-a2bf-4bf5-bb64-c21aa41d97c7 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:53.121782 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba881cf9-3cec-43fe-97c8-efa91f16fb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:53.122446 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba881cf9-3cec-43fe-97c8-efa91f16fb6e admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:53.122815 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba881cf9-3cec-43fe-97c8-efa91f16fb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:53.123199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba881cf9-3cec-43fe-97c8-efa91f16fb6e admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:53.123831 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba881cf9-3cec-43fe-97c8-efa91f16fb6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:53.125789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba881cf9-3cec-43fe-97c8-efa91f16fb6e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:53.164240 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 799/1583] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:00:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:00:53.165276 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4955b-7f1d-4a9b-9d6e-ffe2ed785032 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:53.175781 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4955b-7f1d-4a9b-9d6e-ffe2ed785032 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:53.180220 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee1f5b6-3c17-40f3-b3eb-311e7dc58505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:53.180799 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1f5b6-3c17-40f3-b3eb-311e7dc58505 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:53.181066 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1f5b6-3c17-40f3-b3eb-311e7dc58505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:53.181328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1f5b6-3c17-40f3-b3eb-311e7dc58505 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:53.181997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee1f5b6-3c17-40f3-b3eb-311e7dc58505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:53.183597 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee1f5b6-3c17-40f3-b3eb-311e7dc58505 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:53.193422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 785/1584] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:00:53 2022] POST /identity/v3/users => generated 287 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:00:53.193422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba881cf9-3cec-43fe-97c8-efa91f16fb6e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:53.199736 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 800/1585] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:00:53.199736 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee1f5b6-3c17-40f3-b3eb-311e7dc58505 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:53.201121 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba881cf9-3cec-43fe-97c8-efa91f16fb6e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:53.206286 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7dce2-70e2-4b73-9df1-28940f10c82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:53.206910 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7dce2-70e2-4b73-9df1-28940f10c82c admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:53.207205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7dce2-70e2-4b73-9df1-28940f10c82c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:53.207458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7dce2-70e2-4b73-9df1-28940f10c82c admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:53.207959 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee1f5b6-3c17-40f3-b3eb-311e7dc58505 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:53.208131 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7dce2-70e2-4b73-9df1-28940f10c82c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:53.209708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7dce2-70e2-4b73-9df1-28940f10c82c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:53.212682 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb1dfb-f946-40fa-abeb-4297bdd41bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:53.213763 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb1dfb-f946-40fa-abeb-4297bdd41bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:53.214204 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb1dfb-f946-40fa-abeb-4297bdd41bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:53.214714 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb1dfb-f946-40fa-abeb-4297bdd41bf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:53.215838 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb1dfb-f946-40fa-abeb-4297bdd41bf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:53.218808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb1dfb-f946-40fa-abeb-4297bdd41bf9 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:53.220750 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 786/1586] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:00:53.221672 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7dce2-70e2-4b73-9df1-28940f10c82c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:53.231491 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7dce2-70e2-4b73-9df1-28940f10c82c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:53.239838 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 801/1587] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:00:53.239995 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee45fda-5e0b-4c27-a5c4-8972ce32edbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:53.240099 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb1dfb-f946-40fa-abeb-4297bdd41bf9 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:53.240177 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee45fda-5e0b-4c27-a5c4-8972ce32edbd admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:53.240177 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee45fda-5e0b-4c27-a5c4-8972ce32edbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:53.240177 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee45fda-5e0b-4c27-a5c4-8972ce32edbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:53.240177 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee45fda-5e0b-4c27-a5c4-8972ce32edbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:53.241214 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee45fda-5e0b-4c27-a5c4-8972ce32edbd admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:53.247636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb1dfb-f946-40fa-abeb-4297bdd41bf9 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:53.253244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8d1c1-59c3-4905-ba77-9cf4353c55c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:53.256653 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 787/1588] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:00:53.256790 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8d1c1-59c3-4905-ba77-9cf4353c55c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:53.256790 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8d1c1-59c3-4905-ba77-9cf4353c55c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:53.256790 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8d1c1-59c3-4905-ba77-9cf4353c55c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:53.257044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee45fda-5e0b-4c27-a5c4-8972ce32edbd admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:53.257129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8d1c1-59c3-4905-ba77-9cf4353c55c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:53.257129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8d1c1-59c3-4905-ba77-9cf4353c55c5 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:53.266980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee45fda-5e0b-4c27-a5c4-8972ce32edbd admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:53.267939 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 802/1589] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:00:53.268751 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8d1c1-59c3-4905-ba77-9cf4353c55c5 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:53.274541 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385e6c1-dea2-4a5a-875a-65bd29cd04ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:53.275351 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385e6c1-dea2-4a5a-875a-65bd29cd04ae admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:53.275622 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385e6c1-dea2-4a5a-875a-65bd29cd04ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:53.276448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385e6c1-dea2-4a5a-875a-65bd29cd04ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:53.277118 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385e6c1-dea2-4a5a-875a-65bd29cd04ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:53.278727 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385e6c1-dea2-4a5a-875a-65bd29cd04ae admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:53.279426 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8d1c1-59c3-4905-ba77-9cf4353c55c5 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:53.284103 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fcd20-564f-491c-afe6-71735edc33c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:53.284686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fcd20-564f-491c-afe6-71735edc33c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:53.284961 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fcd20-564f-491c-afe6-71735edc33c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:53.285230 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fcd20-564f-491c-afe6-71735edc33c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:53.285853 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4fcd20-564f-491c-afe6-71735edc33c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:53.287759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4fcd20-564f-491c-afe6-71735edc33c5 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:53.291855 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 788/1590] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:00:53.291855 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385e6c1-dea2-4a5a-875a-65bd29cd04ae admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:53.300987 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385e6c1-dea2-4a5a-875a-65bd29cd04ae admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:53.305574 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a11f2-dfc0-4f68-a0ab-fb9f83851009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:53.306167 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a11f2-dfc0-4f68-a0ab-fb9f83851009 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:53.306439 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a11f2-dfc0-4f68-a0ab-fb9f83851009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:53.306734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a11f2-dfc0-4f68-a0ab-fb9f83851009 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:53.307484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26a11f2-dfc0-4f68-a0ab-fb9f83851009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:53.309062 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26a11f2-dfc0-4f68-a0ab-fb9f83851009 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:53.319918 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 789/1591] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:00:53.320845 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a11f2-dfc0-4f68-a0ab-fb9f83851009 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:53.330677 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a11f2-dfc0-4f68-a0ab-fb9f83851009 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:53.335132 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb787a-e8cf-4bdb-8bdc-d842c171274b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:53.335676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb787a-e8cf-4bdb-8bdc-d842c171274b admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:53.335950 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb787a-e8cf-4bdb-8bdc-d842c171274b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:53.336313 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb787a-e8cf-4bdb-8bdc-d842c171274b admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:53.336917 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb787a-e8cf-4bdb-8bdc-d842c171274b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:53.338438 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb787a-e8cf-4bdb-8bdc-d842c171274b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:53.349208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 790/1592] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:00:53.350027 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb787a-e8cf-4bdb-8bdc-d842c171274b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:53.359682 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb787a-e8cf-4bdb-8bdc-d842c171274b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:53.364393 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24275b24-ee30-4e7a-ab68-5af4b0bc26b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:53.364967 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24275b24-ee30-4e7a-ab68-5af4b0bc26b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:53.365240 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24275b24-ee30-4e7a-ab68-5af4b0bc26b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:53.365508 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24275b24-ee30-4e7a-ab68-5af4b0bc26b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:53.366124 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24275b24-ee30-4e7a-ab68-5af4b0bc26b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:53.367697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24275b24-ee30-4e7a-ab68-5af4b0bc26b7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:53.376786 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 803/1593] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:00:53 2022] POST /identity/v3/projects => generated 363 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:00:53.376786 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fcd20-564f-491c-afe6-71735edc33c5 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:53.383404 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 791/1594] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:00:53.383404 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24275b24-ee30-4e7a-ab68-5af4b0bc26b7 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:53.391238 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24275b24-ee30-4e7a-ab68-5af4b0bc26b7 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:53.439721 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fcd20-564f-491c-afe6-71735edc33c5 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:53.444609 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9562f9a-deb7-49c0-a236-e5784619ea6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:53.445205 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9562f9a-deb7-49c0-a236-e5784619ea6f admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:53.445479 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9562f9a-deb7-49c0-a236-e5784619ea6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:53.445744 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9562f9a-deb7-49c0-a236-e5784619ea6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:53.446384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9562f9a-deb7-49c0-a236-e5784619ea6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:53.448003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9562f9a-deb7-49c0-a236-e5784619ea6f admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:53.451011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b10fa-7cdb-43a9-b289-a06b89015573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:53.451614 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b10fa-7cdb-43a9-b289-a06b89015573 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:53.451893 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b10fa-7cdb-43a9-b289-a06b89015573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:53.452164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b10fa-7cdb-43a9-b289-a06b89015573 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:53.452795 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b10fa-7cdb-43a9-b289-a06b89015573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:53.454486 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b10fa-7cdb-43a9-b289-a06b89015573 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:53.457932 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 804/1595] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:00:53.458814 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9562f9a-deb7-49c0-a236-e5784619ea6f admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:53.464766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 792/1596] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:00:53.465431 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b10fa-7cdb-43a9-b289-a06b89015573 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:53.473276 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9562f9a-deb7-49c0-a236-e5784619ea6f admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:53.475489 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b10fa-7cdb-43a9-b289-a06b89015573 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:53.478132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58efda53-988d-4993-9ed9-30368d17c5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:53.478800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efda53-988d-4993-9ed9-30368d17c5e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:53.479079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efda53-988d-4993-9ed9-30368d17c5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:53.479367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efda53-988d-4993-9ed9-30368d17c5e7 admin admin] PATH_INFO: `/v3/projects/b0b35b1c14ba4a2e9b23a223628b449c/users/b8ceeb13c4c24ceb9ff1f27b4e45f62c/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:53.480260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abbdca0-fd6d-4a33-b387-d61e522c1d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:53.480830 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbdca0-fd6d-4a33-b387-d61e522c1d00 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:53.481112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbdca0-fd6d-4a33-b387-d61e522c1d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:53.481389 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbdca0-fd6d-4a33-b387-d61e522c1d00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:53.482008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abbdca0-fd6d-4a33-b387-d61e522c1d00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:53.483620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abbdca0-fd6d-4a33-b387-d61e522c1d00 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:53.493694 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 793/1597] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:00:53.494276 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58efda53-988d-4993-9ed9-30368d17c5e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0b35b1c14ba4a2e9b23a223628b449c, user_id=b8ceeb13c4c24ceb9ff1f27b4e45f62c, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:53.494457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abbdca0-fd6d-4a33-b387-d61e522c1d00 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:53.497179 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58efda53-988d-4993-9ed9-30368d17c5e7 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:53.504567 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abbdca0-fd6d-4a33-b387-d61e522c1d00 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:53.509486 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01459900-c887-46be-b500-aec8cccfba84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:53.510121 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01459900-c887-46be-b500-aec8cccfba84 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:53.511413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01459900-c887-46be-b500-aec8cccfba84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:53.511413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01459900-c887-46be-b500-aec8cccfba84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:53.511413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01459900-c887-46be-b500-aec8cccfba84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:53.512883 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01459900-c887-46be-b500-aec8cccfba84 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:53.518233 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:00:53.523482 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 794/1598] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:00:53.524782 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01459900-c887-46be-b500-aec8cccfba84 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:53.537149 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01459900-c887-46be-b500-aec8cccfba84 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:53.543672 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 805/1599] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:00:53 2022] PUT /identity/v3/projects/b0b35b1c14ba4a2e9b23a223628b449c/users/b8ceeb13c4c24ceb9ff1f27b4e45f62c/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:00:53.545208 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58efda53-988d-4993-9ed9-30368d17c5e7 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:53.567492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:00:53.598160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9f4c3-bcdd-48c9-916a-036e16412956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:53.598881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9f4c3-bcdd-48c9-916a-036e16412956 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:53.599152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9f4c3-bcdd-48c9-916a-036e16412956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:53.599425 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9f4c3-bcdd-48c9-916a-036e16412956 admin admin] PATH_INFO: `/v3/projects/c748f4142cf54207a35d5d39b6cac877/users/640a5815b1a545fc952ac67f553b6797/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:53.613571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58efda53-988d-4993-9ed9-30368d17c5e7 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:53.613952 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f9f4c3-bcdd-48c9-916a-036e16412956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c748f4142cf54207a35d5d39b6cac877, user_id=640a5815b1a545fc952ac67f553b6797, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:53.616722 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f9f4c3-bcdd-48c9-916a-036e16412956 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:53.618890 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e769d-70d8-454e-995c-c220b6ab8662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:53.619621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e769d-70d8-454e-995c-c220b6ab8662 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:53.619962 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e769d-70d8-454e-995c-c220b6ab8662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:53.626965 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e769d-70d8-454e-995c-c220b6ab8662 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:53.626965 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e769d-70d8-454e-995c-c220b6ab8662 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:53.626965 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e769d-70d8-454e-995c-c220b6ab8662 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:53.709298 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 806/1600] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:00:53 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:00:53.709661 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 795/1601] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:00:53 2022] PUT /identity/v3/projects/c748f4142cf54207a35d5d39b6cac877/users/640a5815b1a545fc952ac67f553b6797/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:00:53.710109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e769d-70d8-454e-995c-c220b6ab8662 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:53.710961 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9f4c3-bcdd-48c9-916a-036e16412956 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:53.723425 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e769d-70d8-454e-995c-c220b6ab8662 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:53.779893 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9f4c3-bcdd-48c9-916a-036e16412956 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:53.783065 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c877c8-6bbd-45df-811d-f535fdf9b975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:53.783667 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c877c8-6bbd-45df-811d-f535fdf9b975 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:53.783940 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c877c8-6bbd-45df-811d-f535fdf9b975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:53.784207 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c877c8-6bbd-45df-811d-f535fdf9b975 admin admin] PATH_INFO: `/v3/projects/61a75581f5eb4854959399c3b2d3a3fc/users/7a4d4018f1f24a3c9b9172607545ad28/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:53.784978 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13479ed2-920b-482b-865e-6eedb47e37f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:53.785624 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13479ed2-920b-482b-865e-6eedb47e37f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:53.785904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13479ed2-920b-482b-865e-6eedb47e37f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:53.786169 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13479ed2-920b-482b-865e-6eedb47e37f6 admin admin] PATH_INFO: `/v3/projects/41cfe24bec5d4adaa1c42c19634fe164/users/2486a6099f3e46e1bdfc81dbc5b640a4/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:53.799953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c877c8-6bbd-45df-811d-f535fdf9b975 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61a75581f5eb4854959399c3b2d3a3fc, user_id=7a4d4018f1f24a3c9b9172607545ad28, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:53.801980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13479ed2-920b-482b-865e-6eedb47e37f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41cfe24bec5d4adaa1c42c19634fe164, user_id=2486a6099f3e46e1bdfc81dbc5b640a4, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:53.802812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c877c8-6bbd-45df-811d-f535fdf9b975 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:53.804654 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13479ed2-920b-482b-865e-6eedb47e37f6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:53.875208 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 807/1602] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:00:53 2022] PUT /identity/v3/projects/61a75581f5eb4854959399c3b2d3a3fc/users/7a4d4018f1f24a3c9b9172607545ad28/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:00:53.876102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c877c8-6bbd-45df-811d-f535fdf9b975 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:53.892536 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 796/1603] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:00:53 2022] PUT /identity/v3/projects/41cfe24bec5d4adaa1c42c19634fe164/users/2486a6099f3e46e1bdfc81dbc5b640a4/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:00:53.893345 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13479ed2-920b-482b-865e-6eedb47e37f6 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:53.942261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c877c8-6bbd-45df-811d-f535fdf9b975 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:53.957534 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13479ed2-920b-482b-865e-6eedb47e37f6 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:53.962326 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e38c1-f058-4962-a22e-f5400b262455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:53.963009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e38c1-f058-4962-a22e-f5400b262455 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:53.963330 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e38c1-f058-4962-a22e-f5400b262455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:53.963641 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e38c1-f058-4962-a22e-f5400b262455 admin admin] PATH_INFO: `/v3/projects/65f0209a8e48496c8d88c885f8364cbd/users/9950f578162449c0845c3aa4aab692d4/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:53.975383 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:00:53.975756 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:00:53.979987 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e38c1-f058-4962-a22e-f5400b262455 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65f0209a8e48496c8d88c885f8364cbd, user_id=9950f578162449c0845c3aa4aab692d4, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:53.982676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e38c1-f058-4962-a22e-f5400b262455 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:53.999256 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef71fb-4927-4ac7-9039-d139d1d84cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:53.999909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef71fb-4927-4ac7-9039-d139d1d84cec admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:54.000183 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef71fb-4927-4ac7-9039-d139d1d84cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:54.000452 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef71fb-4927-4ac7-9039-d139d1d84cec admin admin] PATH_INFO: `/v3/projects/1c3ed6ff48b44385acef77c00e631b6e/users/a66188ce11bc403d9c657efd78f3019a/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:54.015109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef71fb-4927-4ac7-9039-d139d1d84cec admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c3ed6ff48b44385acef77c00e631b6e, user_id=a66188ce11bc403d9c657efd78f3019a, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:54.018250 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef71fb-4927-4ac7-9039-d139d1d84cec admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:54.098558 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 797/1604] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:00:53 2022] PUT /identity/v3/projects/65f0209a8e48496c8d88c885f8364cbd/users/9950f578162449c0845c3aa4aab692d4/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:00:54.099332 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e38c1-f058-4962-a22e-f5400b262455 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:54.120879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 808/1605] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:00:53 2022] PUT /identity/v3/projects/1c3ed6ff48b44385acef77c00e631b6e/users/a66188ce11bc403d9c657efd78f3019a/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:00:54.122189 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc4157-e107-4c9c-a72b-9f03845ca020 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:54.122471 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc4157-e107-4c9c-a72b-9f03845ca020 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:54.122762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc4157-e107-4c9c-a72b-9f03845ca020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:54.142181 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a6cc4157-e107-4c9c-a72b-9f03845ca020 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:54.143811 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a6cc4157-e107-4c9c-a72b-9f03845ca020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:54.161605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e38c1-f058-4962-a22e-f5400b262455 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:54.206868 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 809/1606] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:00:54.207727 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc4157-e107-4c9c-a72b-9f03845ca020 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:54.221960 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf86834-a4eb-4ecc-99c8-6bca1b89c143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:54.223187 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf86834-a4eb-4ecc-99c8-6bca1b89c143 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:54.223464 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf86834-a4eb-4ecc-99c8-6bca1b89c143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:54.223916 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf86834-a4eb-4ecc-99c8-6bca1b89c143 admin admin] PATH_INFO: `/v3/projects/ce035fd37c1549b7a3edaf65fa2f0f30/users/a6c4688db12d420492c64ce3c4b29828/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:54.225886 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc4157-e107-4c9c-a72b-9f03845ca020 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:54.231066 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5130dbd-e057-468b-8c4b-8137c9fde107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:54.231703 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5130dbd-e057-468b-8c4b-8137c9fde107 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:54.231980 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5130dbd-e057-468b-8c4b-8137c9fde107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:54.232253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5130dbd-e057-468b-8c4b-8137c9fde107 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:54.232897 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5130dbd-e057-468b-8c4b-8137c9fde107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:54.234498 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5130dbd-e057-468b-8c4b-8137c9fde107 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:54.239483 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf86834-a4eb-4ecc-99c8-6bca1b89c143 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce035fd37c1549b7a3edaf65fa2f0f30, user_id=a6c4688db12d420492c64ce3c4b29828, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:54.242335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf86834-a4eb-4ecc-99c8-6bca1b89c143 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:54.244581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 810/1607] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:00:54.247873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef9f97-6d16-49e3-9d4e-6a2391dea231 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:54.247873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef9f97-6d16-49e3-9d4e-6a2391dea231 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:54.247873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef9f97-6d16-49e3-9d4e-6a2391dea231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:54.267945 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-ecef9f97-6d16-49e3-9d4e-6a2391dea231 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:54.269130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ecef9f97-6d16-49e3-9d4e-6a2391dea231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:54.276790 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 811/1608] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:00:54.278332 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9ec1f-3234-4990-878b-75242b4b87a0 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:54.278590 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9ec1f-3234-4990-878b-75242b4b87a0 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:54.278915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9ec1f-3234-4990-878b-75242b4b87a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:54.298652 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 798/1609] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:00:54 2022] PUT /identity/v3/projects/ce035fd37c1549b7a3edaf65fa2f0f30/users/a6c4688db12d420492c64ce3c4b29828/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:00:54.300000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff6416-cd66-4498-8e7b-fd1e60656aed None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:54.300272 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff6416-cd66-4498-8e7b-fd1e60656aed None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:54.300572 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff6416-cd66-4498-8e7b-fd1e60656aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:54.319962 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-1bb9ec1f-3234-4990-878b-75242b4b87a0 None None] MFA Rules not processed for user `7a4d4018f1f24a3c9b9172607545ad28`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:54.321174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1bb9ec1f-3234-4990-878b-75242b4b87a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:54.336019 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-01ff6416-cd66-4498-8e7b-fd1e60656aed None None] MFA Rules not processed for user `2486a6099f3e46e1bdfc81dbc5b640a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:54.337197 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-01ff6416-cd66-4498-8e7b-fd1e60656aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:54.386082 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 812/1610] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:00:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:00:54.391486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaea2f1-66aa-47a2-8c9f-d0bd2eed9ead None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:54.391486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaea2f1-66aa-47a2-8c9f-d0bd2eed9ead None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:54.391486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaea2f1-66aa-47a2-8c9f-d0bd2eed9ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:54.404852 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 799/1611] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:00:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:00:54.406220 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b214a74-4e14-4c0d-af0e-ff11d2e37eab None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:54.406514 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b214a74-4e14-4c0d-af0e-ff11d2e37eab None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:54.406969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b214a74-4e14-4c0d-af0e-ff11d2e37eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:54.408108 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-0aaea2f1-66aa-47a2-8c9f-d0bd2eed9ead None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:54.409708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0aaea2f1-66aa-47a2-8c9f-d0bd2eed9ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:54.433087 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1b214a74-4e14-4c0d-af0e-ff11d2e37eab None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:54.434429 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1b214a74-4e14-4c0d-af0e-ff11d2e37eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:54.440209 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:00:54.447152 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:00:54.478461 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 813/1612] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:00:54.479345 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaea2f1-66aa-47a2-8c9f-d0bd2eed9ead None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:54.507267 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 800/1613] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:00:54.508343 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b214a74-4e14-4c0d-af0e-ff11d2e37eab None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:54.518914 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b214a74-4e14-4c0d-af0e-ff11d2e37eab None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:54.523592 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f3bcb-32f4-4bc0-820b-7751ff849aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:54.529411 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f3bcb-32f4-4bc0-820b-7751ff849aa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:54.529411 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f3bcb-32f4-4bc0-820b-7751ff849aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:54.529411 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f3bcb-32f4-4bc0-820b-7751ff849aa7 admin admin] PATH_INFO: `/v3/projects/d150aba7269948e7b02a177c7d91fe5e/users/324191df716e499e9ffcb07d5fe8dd1a/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:54.539716 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f3bcb-32f4-4bc0-820b-7751ff849aa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d150aba7269948e7b02a177c7d91fe5e, user_id=324191df716e499e9ffcb07d5fe8dd1a, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:54.540800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaea2f1-66aa-47a2-8c9f-d0bd2eed9ead None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:54.543726 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f3bcb-32f4-4bc0-820b-7751ff849aa7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:54.544450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59258d0-a40b-4708-be4c-980fcc6aa00d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:54.545051 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59258d0-a40b-4708-be4c-980fcc6aa00d service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:54.545323 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59258d0-a40b-4708-be4c-980fcc6aa00d service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:54.545589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59258d0-a40b-4708-be4c-980fcc6aa00d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:54.554964 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59258d0-a40b-4708-be4c-980fcc6aa00d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:54.556382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59258d0-a40b-4708-be4c-980fcc6aa00d service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:54.566176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 814/1614] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:00:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:00:54.567207 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59258d0-a40b-4708-be4c-980fcc6aa00d service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:54.568781 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a59258d0-a40b-4708-be4c-980fcc6aa00d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:54.579434 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59258d0-a40b-4708-be4c-980fcc6aa00d service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:54.583358 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b72ab0a-5cc2-4b04-befb-1f3fe0e4e115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:54.583931 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b72ab0a-5cc2-4b04-befb-1f3fe0e4e115 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:54.584202 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b72ab0a-5cc2-4b04-befb-1f3fe0e4e115 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:54.584465 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b72ab0a-5cc2-4b04-befb-1f3fe0e4e115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:54.593471 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b72ab0a-5cc2-4b04-befb-1f3fe0e4e115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:54.595046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b72ab0a-5cc2-4b04-befb-1f3fe0e4e115 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:54.604584 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 815/1615] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:00:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:00:54.605780 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42828c5-9b4f-4ce9-b233-ff2fd301bea8 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:54.606059 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42828c5-9b4f-4ce9-b233-ff2fd301bea8 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:54.606403 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42828c5-9b4f-4ce9-b233-ff2fd301bea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:54.641354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-e42828c5-9b4f-4ce9-b233-ff2fd301bea8 None None] MFA Rules not processed for user `a6c4688db12d420492c64ce3c4b29828`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:54.643163 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e42828c5-9b4f-4ce9-b233-ff2fd301bea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:54.664706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 801/1616] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:00:54 2022] PUT /identity/v3/projects/d150aba7269948e7b02a177c7d91fe5e/users/324191df716e499e9ffcb07d5fe8dd1a/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:00:54.666035 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0369b2-a7a5-49e1-815e-3456e91d10ae None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:54.666320 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0369b2-a7a5-49e1-815e-3456e91d10ae None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:54.667999 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0369b2-a7a5-49e1-815e-3456e91d10ae None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:54.667999 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 802/1617] 10.222.0.17 () {54 vars in 962 bytes} [Sun Sep 11 00:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:00:54.668924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9e70c-91bc-45c2-8ef2-2a0bebe87393 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:54.669204 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9e70c-91bc-45c2-8ef2-2a0bebe87393 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:54.669477 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9e70c-91bc-45c2-8ef2-2a0bebe87393 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:54.670804 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 803/1618] 10.222.0.17 () {54 vars in 962 bytes} [Sun Sep 11 00:00:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:00:54.671636 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9e70c-91bc-45c2-8ef2-2a0bebe87393 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:54.723933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 816/1619] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:00:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:00:54.725775 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42828c5-9b4f-4ce9-b233-ff2fd301bea8 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:54.737223 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9e70c-91bc-45c2-8ef2-2a0bebe87393 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:54.738929 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42828c5-9b4f-4ce9-b233-ff2fd301bea8 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:54.740237 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a5da2-3664-4595-aa3a-9dadcb3305f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:54.741159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a5da2-3664-4595-aa3a-9dadcb3305f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:54.741687 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a5da2-3664-4595-aa3a-9dadcb3305f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:54.742054 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a5da2-3664-4595-aa3a-9dadcb3305f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:54.744202 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540e749-87db-4fd8-bd25-0981d3d1c16a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:54.745155 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540e749-87db-4fd8-bd25-0981d3d1c16a service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:54.745584 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540e749-87db-4fd8-bd25-0981d3d1c16a service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:54.746148 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540e749-87db-4fd8-bd25-0981d3d1c16a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:54.812141 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a5da2-3664-4595-aa3a-9dadcb3305f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:54.813584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a5da2-3664-4595-aa3a-9dadcb3305f1 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:54.824802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 804/1620] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:00:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:00:54.825655 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a5da2-3664-4595-aa3a-9dadcb3305f1 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:54.826928 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c540e749-87db-4fd8-bd25-0981d3d1c16a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:54.828716 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c540e749-87db-4fd8-bd25-0981d3d1c16a service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:54.835828 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a5da2-3664-4595-aa3a-9dadcb3305f1 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:54.840990 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557747ca-c134-4ea3-87e8-23de4441211e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:54.841738 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557747ca-c134-4ea3-87e8-23de4441211e admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:54.842025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557747ca-c134-4ea3-87e8-23de4441211e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:54.842297 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557747ca-c134-4ea3-87e8-23de4441211e admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:54.842995 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557747ca-c134-4ea3-87e8-23de4441211e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:54.844744 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 817/1621] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:00:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:00:54.845409 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557747ca-c134-4ea3-87e8-23de4441211e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:54.846896 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572122f-6542-4f39-b293-40ae5ab301db None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:54.847178 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572122f-6542-4f39-b293-40ae5ab301db None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:54.847446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572122f-6542-4f39-b293-40ae5ab301db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:54.859392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 805/1622] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:00:54.859392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc9330-0040-46ec-b993-e0b028fb4745 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:54.859392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc9330-0040-46ec-b993-e0b028fb4745 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:54.859392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc9330-0040-46ec-b993-e0b028fb4745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:54.896284 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-1572122f-6542-4f39-b293-40ae5ab301db None None] MFA Rules not processed for user `636bfafb693044d283530063047fc0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:54.897476 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1572122f-6542-4f39-b293-40ae5ab301db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:54.904349 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-bfcc9330-0040-46ec-b993-e0b028fb4745 None None] MFA Rules not processed for user `636bfafb693044d283530063047fc0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:54.906357 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-bfcc9330-0040-46ec-b993-e0b028fb4745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:54.984088 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 806/1623] 10.222.0.17 () {58 vars in 1059 bytes} [Sun Sep 11 00:00:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:00:54.984934 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcc9330-0040-46ec-b993-e0b028fb4745 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:55.004455 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 818/1624] 10.222.0.17 () {58 vars in 1059 bytes} [Sun Sep 11 00:00:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:00:55.008280 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcc9330-0040-46ec-b993-e0b028fb4745 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:55.012307 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1572122f-6542-4f39-b293-40ae5ab301db None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:55.013447 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25769188-4f14-4c4a-bd81-6b840a764c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:55.014190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25769188-4f14-4c4a-bd81-6b840a764c2f admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:55.014562 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25769188-4f14-4c4a-bd81-6b840a764c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:55.014965 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25769188-4f14-4c4a-bd81-6b840a764c2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:55.016162 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25769188-4f14-4c4a-bd81-6b840a764c2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:55.018716 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25769188-4f14-4c4a-bd81-6b840a764c2f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:55.022943 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1572122f-6542-4f39-b293-40ae5ab301db None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:55.027983 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768e1db-8e5f-45cf-ad41-dafa8d0eb0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:55.028645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768e1db-8e5f-45cf-ad41-dafa8d0eb0d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:55.028916 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768e1db-8e5f-45cf-ad41-dafa8d0eb0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:55.029183 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768e1db-8e5f-45cf-ad41-dafa8d0eb0d8 admin admin] PATH_INFO: `/v3/projects/d150aba7269948e7b02a177c7d91fe5e/users/324191df716e499e9ffcb07d5fe8dd1a/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:55.031285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768e1db-8e5f-45cf-ad41-dafa8d0eb0d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d150aba7269948e7b02a177c7d91fe5e, user_id=324191df716e499e9ffcb07d5fe8dd1a, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:55.035124 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768e1db-8e5f-45cf-ad41-dafa8d0eb0d8 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:55.061603 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 819/1625] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:00:55 2022] PUT /identity/v3/projects/d150aba7269948e7b02a177c7d91fe5e/users/324191df716e499e9ffcb07d5fe8dd1a/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:00:55.061603 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768e1db-8e5f-45cf-ad41-dafa8d0eb0d8 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:55.122414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 807/1626] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:00:54 2022] POST /identity/v3/projects => generated 371 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:00:55.122414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25769188-4f14-4c4a-bd81-6b840a764c2f admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:55.158194 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768e1db-8e5f-45cf-ad41-dafa8d0eb0d8 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:55.257319 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347a4f6-7666-40e0-9e5f-a3886c9a2781 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:55.257988 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25769188-4f14-4c4a-bd81-6b840a764c2f admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:55.259128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347a4f6-7666-40e0-9e5f-a3886c9a2781 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:55.259725 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347a4f6-7666-40e0-9e5f-a3886c9a2781 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:55.260253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347a4f6-7666-40e0-9e5f-a3886c9a2781 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:55.268139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5e194-e1fb-4b24-8a78-ccb1ad6d07c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:55.268766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5e194-e1fb-4b24-8a78-ccb1ad6d07c5 service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:55.269060 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5e194-e1fb-4b24-8a78-ccb1ad6d07c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:55.271160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5e194-e1fb-4b24-8a78-ccb1ad6d07c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:55.337275 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5e194-e1fb-4b24-8a78-ccb1ad6d07c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:55.341206 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5e194-e1fb-4b24-8a78-ccb1ad6d07c5 service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:55.347604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347a4f6-7666-40e0-9e5f-a3886c9a2781 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:55.348678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 808/1627] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:00:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:00:55.349165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347a4f6-7666-40e0-9e5f-a3886c9a2781 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:55.349929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f530b-a9c9-49e2-81a6-ad869b541a87 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:55.350206 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f530b-a9c9-49e2-81a6-ad869b541a87 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:55.350472 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f530b-a9c9-49e2-81a6-ad869b541a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:55.358534 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 820/1628] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:00:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:00:55.359502 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347a4f6-7666-40e0-9e5f-a3886c9a2781 service nova] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:55.410439 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-329f530b-a9c9-49e2-81a6-ad869b541a87 None None] MFA Rules not processed for user `324191df716e499e9ffcb07d5fe8dd1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:55.413875 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-329f530b-a9c9-49e2-81a6-ad869b541a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:55.460065 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347a4f6-7666-40e0-9e5f-a3886c9a2781 service nova] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:55.470918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec5b5e-cc5e-4904-8c97-6c78b41c464d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:55.471540 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec5b5e-cc5e-4904-8c97-6c78b41c464d admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:55.471820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec5b5e-cc5e-4904-8c97-6c78b41c464d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:55.472629 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec5b5e-cc5e-4904-8c97-6c78b41c464d admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:55.473281 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec5b5e-cc5e-4904-8c97-6c78b41c464d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:55.476402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec5b5e-cc5e-4904-8c97-6c78b41c464d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:55.566689 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 809/1629] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:00:55 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:00:55.580394 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f530b-a9c9-49e2-81a6-ad869b541a87 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:55.599351 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 821/1630] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:00:55 2022] POST /identity/v3/users => generated 292 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:00:55.602723 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec5b5e-cc5e-4904-8c97-6c78b41c464d admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:55.636852 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec5b5e-cc5e-4904-8c97-6c78b41c464d admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:55.653204 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f33f9-9d2d-49da-a19c-b2380c22792f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:55.656245 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f33f9-9d2d-49da-a19c-b2380c22792f admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:55.656245 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f33f9-9d2d-49da-a19c-b2380c22792f admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:55.656245 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f33f9-9d2d-49da-a19c-b2380c22792f admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:55.658344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205f33f9-9d2d-49da-a19c-b2380c22792f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:55.662313 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205f33f9-9d2d-49da-a19c-b2380c22792f admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:55.686599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 822/1631] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:00:55.688012 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a15c3-a110-4445-8f65-78d1cb79146a None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:55.688301 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a15c3-a110-4445-8f65-78d1cb79146a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:55.689116 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a15c3-a110-4445-8f65-78d1cb79146a None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:55.695138 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 823/1632] 10.222.0.17 () {54 vars in 912 bytes} [Sun Sep 11 00:00:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:00:55.695972 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a15c3-a110-4445-8f65-78d1cb79146a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:55.703586 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f530b-a9c9-49e2-81a6-ad869b541a87 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:55.706237 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:00:55.709780 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5e341-ec39-46be-96cf-f52646f0742b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:55.711639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5e341-ec39-46be-96cf-f52646f0742b service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:55.711639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5e341-ec39-46be-96cf-f52646f0742b service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:55.711639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5e341-ec39-46be-96cf-f52646f0742b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:55.724413 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a15c3-a110-4445-8f65-78d1cb79146a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:55.736675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5e341-ec39-46be-96cf-f52646f0742b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:55.742367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d0d9e-2d3f-4549-ac2e-b0f6786fb7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:55.742367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d0d9e-2d3f-4549-ac2e-b0f6786fb7f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:55.742367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d0d9e-2d3f-4549-ac2e-b0f6786fb7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:55.742367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d0d9e-2d3f-4549-ac2e-b0f6786fb7f8 admin admin] PATH_INFO: `/v3/projects/beee0e3d927642589c064aac5e697ff4/users/e5f59a72ca3c40cba8c14c9193162f7d/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:55.743652 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5e341-ec39-46be-96cf-f52646f0742b service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:55.753016 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 810/1633] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:00:55 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:00:55.755296 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd963415-8cfa-46f5-aea8-7d899020f6d9 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:55.755576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd963415-8cfa-46f5-aea8-7d899020f6d9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:55.755908 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd963415-8cfa-46f5-aea8-7d899020f6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:55.756677 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:00:55.770277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8d0d9e-2d3f-4549-ac2e-b0f6786fb7f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=beee0e3d927642589c064aac5e697ff4, user_id=e5f59a72ca3c40cba8c14c9193162f7d, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:55.774007 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8d0d9e-2d3f-4549-ac2e-b0f6786fb7f8 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:55.805243 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-bd963415-8cfa-46f5-aea8-7d899020f6d9 None None] MFA Rules not processed for user `636bfafb693044d283530063047fc0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:55.807009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-bd963415-8cfa-46f5-aea8-7d899020f6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:55.819140 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 811/1634] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Sep 11 00:00:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:00:55.821623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9f350-d84e-4bbc-8722-e845b5b12d88 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:55.822362 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9f350-d84e-4bbc-8722-e845b5b12d88 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:55.822872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9f350-d84e-4bbc-8722-e845b5b12d88 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:55.824495 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 812/1635] 10.222.0.17 () {54 vars in 912 bytes} [Sun Sep 11 00:00:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:00:55.832482 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ff142-f41c-4874-bff6-2f88ea1b4d03 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:55.832925 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ff142-f41c-4874-bff6-2f88ea1b4d03 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:55.833290 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ff142-f41c-4874-bff6-2f88ea1b4d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:55.857738 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b19ff142-f41c-4874-bff6-2f88ea1b4d03 None None] MFA Rules not processed for user `636bfafb693044d283530063047fc0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:55.858995 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b19ff142-f41c-4874-bff6-2f88ea1b4d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:55.876373 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 824/1636] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:00:55 2022] PUT /identity/v3/projects/beee0e3d927642589c064aac5e697ff4/users/e5f59a72ca3c40cba8c14c9193162f7d/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:00:55.878462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d0d9e-2d3f-4549-ac2e-b0f6786fb7f8 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:55.879862 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 813/1637] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Sep 11 00:00:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:00:55.880704 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ff142-f41c-4874-bff6-2f88ea1b4d03 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:55.902589 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ff142-f41c-4874-bff6-2f88ea1b4d03 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:55.967410 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d0d9e-2d3f-4549-ac2e-b0f6786fb7f8 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:55.972406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8379e7-7c90-4946-b8ac-c8808eb5eff8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:55.973012 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8379e7-7c90-4946-b8ac-c8808eb5eff8 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:55.973288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8379e7-7c90-4946-b8ac-c8808eb5eff8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:55.973563 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8379e7-7c90-4946-b8ac-c8808eb5eff8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:55.973856 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a559543f-d705-479f-9978-fdbde404339c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:55.974414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a559543f-d705-479f-9978-fdbde404339c admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:55.974701 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a559543f-d705-479f-9978-fdbde404339c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:55.974975 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a559543f-d705-479f-9978-fdbde404339c admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:55.975619 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a559543f-d705-479f-9978-fdbde404339c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:55.977231 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a559543f-d705-479f-9978-fdbde404339c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:55.994243 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 814/1638] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:00:55.995363 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a559543f-d705-479f-9978-fdbde404339c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:56.006381 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a559543f-d705-479f-9978-fdbde404339c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:56.012598 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42492c-8288-4861-b3de-ce66715513b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:56.013082 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42492c-8288-4861-b3de-ce66715513b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:56.013455 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42492c-8288-4861-b3de-ce66715513b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:56.013737 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42492c-8288-4861-b3de-ce66715513b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:56.062268 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:00:56.080377 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8379e7-7c90-4946-b8ac-c8808eb5eff8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:56.082188 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8379e7-7c90-4946-b8ac-c8808eb5eff8 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:56.098037 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 825/1639] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:00:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:00:56.098448 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8379e7-7c90-4946-b8ac-c8808eb5eff8 service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:56.112575 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8379e7-7c90-4946-b8ac-c8808eb5eff8 service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:56.117992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1ffb5-3da1-44ce-a735-64ea163a7a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:56.118843 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1ffb5-3da1-44ce-a735-64ea163a7a6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:56.119385 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1ffb5-3da1-44ce-a735-64ea163a7a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:56.119808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1ffb5-3da1-44ce-a735-64ea163a7a6f admin admin] PATH_INFO: `/v3/projects/beee0e3d927642589c064aac5e697ff4/users/e5f59a72ca3c40cba8c14c9193162f7d/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:56.128657 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1ffb5-3da1-44ce-a735-64ea163a7a6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=beee0e3d927642589c064aac5e697ff4, user_id=e5f59a72ca3c40cba8c14c9193162f7d, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:56.133091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1ffb5-3da1-44ce-a735-64ea163a7a6f admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:56.152563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af42492c-8288-4861-b3de-ce66715513b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:56.154320 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af42492c-8288-4861-b3de-ce66715513b1 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:56.163603 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 826/1640] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:00:56 2022] PUT /identity/v3/projects/beee0e3d927642589c064aac5e697ff4/users/e5f59a72ca3c40cba8c14c9193162f7d/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:00:56.169629 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 815/1641] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:00:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:00:56.174499 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5f017-2161-4985-82d3-258b14f85e37 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:56.174783 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5f017-2161-4985-82d3-258b14f85e37 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:56.175066 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5f017-2161-4985-82d3-258b14f85e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:56.228198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-5ba5f017-2161-4985-82d3-258b14f85e37 None None] MFA Rules not processed for user `e5f59a72ca3c40cba8c14c9193162f7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:00:56.229425 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5ba5f017-2161-4985-82d3-258b14f85e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:00:56.241048 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:00:56.363141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 827/1642] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:00:56 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:00:56.721370 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:00:57.291978 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:00:57.500596 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42492c-8288-4861-b3de-ce66715513b1 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:57.512418 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42492c-8288-4861-b3de-ce66715513b1 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:57.580725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8f641-4963-443d-975e-4f79ef85dd1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:57.581612 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8f641-4963-443d-975e-4f79ef85dd1e service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:57.582555 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8f641-4963-443d-975e-4f79ef85dd1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:57.582954 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8f641-4963-443d-975e-4f79ef85dd1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:57.595712 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8f641-4963-443d-975e-4f79ef85dd1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:57.597101 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8f641-4963-443d-975e-4f79ef85dd1e service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:57.625061 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 816/1643] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:00:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:00:58.248850 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5f017-2161-4985-82d3-258b14f85e37 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:00:58.274510 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5f017-2161-4985-82d3-258b14f85e37 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:00:58.285658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6997b1d-9ece-4a82-80d3-5b99d2af39ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:00:58.288122 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6997b1d-9ece-4a82-80d3-5b99d2af39ec service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:00:58.288700 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6997b1d-9ece-4a82-80d3-5b99d2af39ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:00:58.289108 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6997b1d-9ece-4a82-80d3-5b99d2af39ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:00:58.396589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:00:58.409941 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6997b1d-9ece-4a82-80d3-5b99d2af39ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:00:58.411277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6997b1d-9ece-4a82-80d3-5b99d2af39ec service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:00:58.444337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 828/1644] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:00:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:01:00.512067 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8f641-4963-443d-975e-4f79ef85dd1e service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:00.705893 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8f641-4963-443d-975e-4f79ef85dd1e service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:00.726731 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cec0bf-89ec-4b40-98ae-955d7be30c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:00.727385 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cec0bf-89ec-4b40-98ae-955d7be30c29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:00.732546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cec0bf-89ec-4b40-98ae-955d7be30c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:00.732546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cec0bf-89ec-4b40-98ae-955d7be30c29 admin admin] PATH_INFO: `/v3/users/454283506bad4c978d1da1fef5813c4a` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:00.732546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cec0bf-89ec-4b40-98ae-955d7be30c29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=454283506bad4c978d1da1fef5813c4a)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:00.742948 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cec0bf-89ec-4b40-98ae-955d7be30c29 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:01.037206 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-92cec0bf-89ec-4b40-98ae-955d7be30c29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '454283506bad4c978d1da1fef5813c4a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:01.066415 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-92cec0bf-89ec-4b40-98ae-955d7be30c29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '454283506bad4c978d1da1fef5813c4a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:01.070293 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:01.102714 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-92cec0bf-89ec-4b40-98ae-955d7be30c29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '454283506bad4c978d1da1fef5813c4a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:01.183869 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-92cec0bf-89ec-4b40-98ae-955d7be30c29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '454283506bad4c978d1da1fef5813c4a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:01.184510 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-92cec0bf-89ec-4b40-98ae-955d7be30c29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '454283506bad4c978d1da1fef5813c4a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:01.201023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 817/1645] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:01:00 2022] DELETE /identity/v3/users/454283506bad4c978d1da1fef5813c4a => generated 0 bytes in 687 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:01.203133 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361eaeb7-07d3-49ba-a0f9-12a24e9f1416 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:01.203621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361eaeb7-07d3-49ba-a0f9-12a24e9f1416 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:01.203958 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361eaeb7-07d3-49ba-a0f9-12a24e9f1416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:01.249762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-361eaeb7-07d3-49ba-a0f9-12a24e9f1416 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:01.249762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-361eaeb7-07d3-49ba-a0f9-12a24e9f1416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:01.376163 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 829/1646] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:01.411288 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cec0bf-89ec-4b40-98ae-955d7be30c29 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:01.413176 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-92cec0bf-89ec-4b40-98ae-955d7be30c29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:01.543736 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cec0bf-89ec-4b40-98ae-955d7be30c29 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:01.554235 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050cc599-6d65-4682-b228-ab77d1f75500 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:01.555024 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cc599-6d65-4682-b228-ab77d1f75500 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:01.555374 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cc599-6d65-4682-b228-ab77d1f75500 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:01.555721 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cc599-6d65-4682-b228-ab77d1f75500 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:01.580406 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050cc599-6d65-4682-b228-ab77d1f75500 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:01.582818 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050cc599-6d65-4682-b228-ab77d1f75500 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:01.603325 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 818/1647] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:01:03.337916 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361eaeb7-07d3-49ba-a0f9-12a24e9f1416 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:03.338865 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-361eaeb7-07d3-49ba-a0f9-12a24e9f1416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:03.362237 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361eaeb7-07d3-49ba-a0f9-12a24e9f1416 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:03.380134 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06218d-99b5-4860-8f42-7bfc3a6dd0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:03.381186 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06218d-99b5-4860-8f42-7bfc3a6dd0cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:03.381608 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06218d-99b5-4860-8f42-7bfc3a6dd0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:03.382414 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06218d-99b5-4860-8f42-7bfc3a6dd0cf admin admin] PATH_INFO: `/v3/projects/0ec8e2b8bb6b4cfbbe9dc3b996700e5f` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:03.384474 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a06218d-99b5-4860-8f42-7bfc3a6dd0cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ec8e2b8bb6b4cfbbe9dc3b996700e5f)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:03.388685 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a06218d-99b5-4860-8f42-7bfc3a6dd0cf admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:03.538991 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-6a06218d-99b5-4860-8f42-7bfc3a6dd0cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ec8e2b8bb6b4cfbbe9dc3b996700e5f', 'request_id': 'req-6a06218d-99b5-4860-8f42-7bfc3a6dd0cf'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:03.649377 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-6a06218d-99b5-4860-8f42-7bfc3a6dd0cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ec8e2b8bb6b4cfbbe9dc3b996700e5f', 'request_id': 'req-6a06218d-99b5-4860-8f42-7bfc3a6dd0cf'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:03.654475 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-6a06218d-99b5-4860-8f42-7bfc3a6dd0cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ec8e2b8bb6b4cfbbe9dc3b996700e5f', 'request_id': 'req-6a06218d-99b5-4860-8f42-7bfc3a6dd0cf'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:03.676649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-6a06218d-99b5-4860-8f42-7bfc3a6dd0cf admin admin] The token cache is being invalidate because project 0ec8e2b8bb6b4cfbbe9dc3b996700e5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:01:03.677990 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-6a06218d-99b5-4860-8f42-7bfc3a6dd0cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:03.678621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-6a06218d-99b5-4860-8f42-7bfc3a6dd0cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:03.685009 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 830/1648] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:01:03 2022] DELETE /identity/v3/projects/0ec8e2b8bb6b4cfbbe9dc3b996700e5f => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:01:03.689025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050cc599-6d65-4682-b228-ab77d1f75500 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:03.707122 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050cc599-6d65-4682-b228-ab77d1f75500 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:03.827537 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc702a2-dda1-4aef-8a6d-f37e0ee7f700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:03.828460 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc702a2-dda1-4aef-8a6d-f37e0ee7f700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:03.829504 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc702a2-dda1-4aef-8a6d-f37e0ee7f700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:03.829891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc702a2-dda1-4aef-8a6d-f37e0ee7f700 admin admin] PATH_INFO: `/v3/users/324191df716e499e9ffcb07d5fe8dd1a` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:03.832985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc702a2-dda1-4aef-8a6d-f37e0ee7f700 admin admin] RBAC: Authorizing `identity:delete_user(user_id=324191df716e499e9ffcb07d5fe8dd1a)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:03.835931 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc702a2-dda1-4aef-8a6d-f37e0ee7f700 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:04.131257 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fbc702a2-dda1-4aef-8a6d-f37e0ee7f700 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '324191df716e499e9ffcb07d5fe8dd1a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:04.143168 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fbc702a2-dda1-4aef-8a6d-f37e0ee7f700 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '324191df716e499e9ffcb07d5fe8dd1a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:04.172974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fbc702a2-dda1-4aef-8a6d-f37e0ee7f700 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '324191df716e499e9ffcb07d5fe8dd1a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:04.281202 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fbc702a2-dda1-4aef-8a6d-f37e0ee7f700 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '324191df716e499e9ffcb07d5fe8dd1a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:04.283864 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fbc702a2-dda1-4aef-8a6d-f37e0ee7f700 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '324191df716e499e9ffcb07d5fe8dd1a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:04.286095 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:04.291979 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 819/1649] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:01:03 2022] DELETE /identity/v3/users/324191df716e499e9ffcb07d5fe8dd1a => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:05.983111 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06218d-99b5-4860-8f42-7bfc3a6dd0cf admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:05.984685 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6a06218d-99b5-4860-8f42-7bfc3a6dd0cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:06.004306 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06218d-99b5-4860-8f42-7bfc3a6dd0cf admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:06.156911 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05f272-e362-42a0-b676-50cbf17f3704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:06.157746 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05f272-e362-42a0-b676-50cbf17f3704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:06.158136 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05f272-e362-42a0-b676-50cbf17f3704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:06.158618 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05f272-e362-42a0-b676-50cbf17f3704 admin admin] PATH_INFO: `/v3/users/2486a6099f3e46e1bdfc81dbc5b640a4` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:06.166274 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c05f272-e362-42a0-b676-50cbf17f3704 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2486a6099f3e46e1bdfc81dbc5b640a4)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:06.167704 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: WARNING py.warnings [None req-0c05f272-e362-42a0-b676-50cbf17f3704 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 00:01:06.167704 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: warnings.warn(msg) Sep 11 00:01:06.167704 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: Sep 11 00:01:06.169161 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c05f272-e362-42a0-b676-50cbf17f3704 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:06.217270 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc702a2-dda1-4aef-8a6d-f37e0ee7f700 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:06.220135 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fbc702a2-dda1-4aef-8a6d-f37e0ee7f700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:06.242256 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc702a2-dda1-4aef-8a6d-f37e0ee7f700 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:06.253969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7304a5-0022-47df-a833-d3d93d0b9784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:06.254374 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7304a5-0022-47df-a833-d3d93d0b9784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:06.254683 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7304a5-0022-47df-a833-d3d93d0b9784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:06.256682 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7304a5-0022-47df-a833-d3d93d0b9784 admin admin] PATH_INFO: `/v3/projects/d150aba7269948e7b02a177c7d91fe5e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:06.258138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7304a5-0022-47df-a833-d3d93d0b9784 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d150aba7269948e7b02a177c7d91fe5e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:06.260001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: WARNING py.warnings [None req-0d7304a5-0022-47df-a833-d3d93d0b9784 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 00:01:06.260001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: warnings.warn(msg) Sep 11 00:01:06.260001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: Sep 11 00:01:06.265907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7304a5-0022-47df-a833-d3d93d0b9784 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:06.271435 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:06.492844 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0d7304a5-0022-47df-a833-d3d93d0b9784 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd150aba7269948e7b02a177c7d91fe5e', 'request_id': 'req-0d7304a5-0022-47df-a833-d3d93d0b9784'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:06.505664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0c05f272-e362-42a0-b676-50cbf17f3704 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2486a6099f3e46e1bdfc81dbc5b640a4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:06.533268 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0d7304a5-0022-47df-a833-d3d93d0b9784 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd150aba7269948e7b02a177c7d91fe5e', 'request_id': 'req-0d7304a5-0022-47df-a833-d3d93d0b9784'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:06.547821 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0d7304a5-0022-47df-a833-d3d93d0b9784 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd150aba7269948e7b02a177c7d91fe5e', 'request_id': 'req-0d7304a5-0022-47df-a833-d3d93d0b9784'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:06.566254 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0c05f272-e362-42a0-b676-50cbf17f3704 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2486a6099f3e46e1bdfc81dbc5b640a4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:06.602924 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0c05f272-e362-42a0-b676-50cbf17f3704 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2486a6099f3e46e1bdfc81dbc5b640a4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:06.605256 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0c05f272-e362-42a0-b676-50cbf17f3704 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2486a6099f3e46e1bdfc81dbc5b640a4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:06.631750 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0d7304a5-0022-47df-a833-d3d93d0b9784 admin admin] The token cache is being invalidate because project d150aba7269948e7b02a177c7d91fe5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:01:06.632145 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0d7304a5-0022-47df-a833-d3d93d0b9784 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:06.634777 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0d7304a5-0022-47df-a833-d3d93d0b9784 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:06.636397 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 820/1650] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:01:06 2022] DELETE /identity/v3/projects/d150aba7269948e7b02a177c7d91fe5e => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:01:06.646206 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba518e-2e9d-41d0-a849-3d57464507b5 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:06.646787 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba518e-2e9d-41d0-a849-3d57464507b5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:06.647340 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba518e-2e9d-41d0-a849-3d57464507b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:06.691612 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-81ba518e-2e9d-41d0-a849-3d57464507b5 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:06.701671 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-81ba518e-2e9d-41d0-a849-3d57464507b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:06.715501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0c05f272-e362-42a0-b676-50cbf17f3704 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2486a6099f3e46e1bdfc81dbc5b640a4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:06.725351 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 831/1651] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:01:05 2022] DELETE /identity/v3/users/2486a6099f3e46e1bdfc81dbc5b640a4 => generated 0 bytes in 743 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:06.729573 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc63272-7407-40a5-b352-db0b98bc586b None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:06.730084 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc63272-7407-40a5-b352-db0b98bc586b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:06.730493 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc63272-7407-40a5-b352-db0b98bc586b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:06.793346 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-4fc63272-7407-40a5-b352-db0b98bc586b None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:06.795794 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4fc63272-7407-40a5-b352-db0b98bc586b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:06.909976 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 821/1652] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:06.919006 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1486d2c-34aa-4030-be4d-0766405dc84b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:06.920668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1486d2c-34aa-4030-be4d-0766405dc84b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:06.920668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1486d2c-34aa-4030-be4d-0766405dc84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:06.963365 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f1486d2c-34aa-4030-be4d-0766405dc84b None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:06.964614 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f1486d2c-34aa-4030-be4d-0766405dc84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:06.981869 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 822/1653] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:01:06.986518 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1486d2c-34aa-4030-be4d-0766405dc84b None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:06.995109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 832/1654] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:07.013427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1486d2c-34aa-4030-be4d-0766405dc84b None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:07.029367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc63272-7407-40a5-b352-db0b98bc586b None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:07.030705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a40fcf-befa-4c63-9ccf-9fe60ed4f8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:07.031328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a40fcf-befa-4c63-9ccf-9fe60ed4f8cb admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:07.031565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4fc63272-7407-40a5-b352-db0b98bc586b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:07.031681 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a40fcf-befa-4c63-9ccf-9fe60ed4f8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:07.032011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a40fcf-befa-4c63-9ccf-9fe60ed4f8cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:07.032682 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a40fcf-befa-4c63-9ccf-9fe60ed4f8cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:01:07.032987 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a40fcf-befa-4c63-9ccf-9fe60ed4f8cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:07.034617 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a40fcf-befa-4c63-9ccf-9fe60ed4f8cb admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:07.066454 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 823/1655] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:01:07.078691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a40fcf-befa-4c63-9ccf-9fe60ed4f8cb admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:07.100517 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a40fcf-befa-4c63-9ccf-9fe60ed4f8cb admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:07.108824 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637c9c9-361f-407b-acc3-f308bb4cd7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:07.109749 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637c9c9-361f-407b-acc3-f308bb4cd7e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:07.110245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637c9c9-361f-407b-acc3-f308bb4cd7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:07.110843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637c9c9-361f-407b-acc3-f308bb4cd7e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:07.111680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637c9c9-361f-407b-acc3-f308bb4cd7e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:07.115179 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637c9c9-361f-407b-acc3-f308bb4cd7e3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:07.205196 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc63272-7407-40a5-b352-db0b98bc586b None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:07.224592 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c08a4-82d6-492b-9996-708a788e1069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:07.225680 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c08a4-82d6-492b-9996-708a788e1069 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:07.226067 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c08a4-82d6-492b-9996-708a788e1069 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:07.226411 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c08a4-82d6-492b-9996-708a788e1069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:07.243787 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c08a4-82d6-492b-9996-708a788e1069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:07.248467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c08a4-82d6-492b-9996-708a788e1069 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:07.262720 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 824/1656] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:01:07 2022] POST /identity/v3/projects => generated 377 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:01:07.268022 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637c9c9-361f-407b-acc3-f308bb4cd7e3 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:07.272923 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 833/1657] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:01:07.368365 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:07.393799 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637c9c9-361f-407b-acc3-f308bb4cd7e3 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:07.403427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54fbfd2-1951-4b0c-a687-59a6b25d8a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:07.404117 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54fbfd2-1951-4b0c-a687-59a6b25d8a22 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:07.404507 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54fbfd2-1951-4b0c-a687-59a6b25d8a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:07.404787 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54fbfd2-1951-4b0c-a687-59a6b25d8a22 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:07.405424 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54fbfd2-1951-4b0c-a687-59a6b25d8a22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:07.407865 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54fbfd2-1951-4b0c-a687-59a6b25d8a22 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:07.532092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 825/1658] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:01:07 2022] POST /identity/v3/users => generated 289 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:01:07.533361 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c08a4-82d6-492b-9996-708a788e1069 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:07.545119 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c08a4-82d6-492b-9996-708a788e1069 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:07.558454 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef51b8-aa47-4173-b9f5-ee1f1a009193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:07.560144 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef51b8-aa47-4173-b9f5-ee1f1a009193 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:07.560442 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef51b8-aa47-4173-b9f5-ee1f1a009193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:07.561659 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef51b8-aa47-4173-b9f5-ee1f1a009193 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:07.562323 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef51b8-aa47-4173-b9f5-ee1f1a009193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:07.566551 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef51b8-aa47-4173-b9f5-ee1f1a009193 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:07.594006 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 834/1659] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:01:07.595743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54fbfd2-1951-4b0c-a687-59a6b25d8a22 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:07.606054 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54fbfd2-1951-4b0c-a687-59a6b25d8a22 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:07.615476 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f3661-02e8-49e7-b849-bad0862c48b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:07.615476 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f3661-02e8-49e7-b849-bad0862c48b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:07.615476 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f3661-02e8-49e7-b849-bad0862c48b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:07.615476 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f3661-02e8-49e7-b849-bad0862c48b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:07.615843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561f3661-02e8-49e7-b849-bad0862c48b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:07.616078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561f3661-02e8-49e7-b849-bad0862c48b5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:07.638544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 826/1660] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:01:07.638650 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef51b8-aa47-4173-b9f5-ee1f1a009193 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:07.643678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51165985-87cc-404d-9b2f-dba2bb45bef3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:07.645483 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51165985-87cc-404d-9b2f-dba2bb45bef3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:07.645773 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51165985-87cc-404d-9b2f-dba2bb45bef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:07.653316 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef51b8-aa47-4173-b9f5-ee1f1a009193 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:07.671808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a173e-e2c1-4540-94bb-8b66dc5b5de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:07.672802 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a173e-e2c1-4540-94bb-8b66dc5b5de1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:07.673074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a173e-e2c1-4540-94bb-8b66dc5b5de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:07.673351 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a173e-e2c1-4540-94bb-8b66dc5b5de1 admin admin] PATH_INFO: `/v3/projects/a9fc207d043e4c19acdf86810c62b55a/users/7e8983c2c24941948a58d3c8838e7851/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:07.695503 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-51165985-87cc-404d-9b2f-dba2bb45bef3 None None] MFA Rules not processed for user `9950f578162449c0845c3aa4aab692d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:07.696779 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-51165985-87cc-404d-9b2f-dba2bb45bef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:07.702017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a173e-e2c1-4540-94bb-8b66dc5b5de1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9fc207d043e4c19acdf86810c62b55a, user_id=7e8983c2c24941948a58d3c8838e7851, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:07.704898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a173e-e2c1-4540-94bb-8b66dc5b5de1 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:07.824831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 835/1661] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:01:07 2022] PUT /identity/v3/projects/a9fc207d043e4c19acdf86810c62b55a/users/7e8983c2c24941948a58d3c8838e7851/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:01:07.836047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7897f9-d2bd-4788-839a-4041ca97c43c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:07.836683 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7897f9-d2bd-4788-839a-4041ca97c43c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:07.837115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7897f9-d2bd-4788-839a-4041ca97c43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:07.896500 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:07.928530 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 827/1662] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:07.935508 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51165985-87cc-404d-9b2f-dba2bb45bef3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:07.943148 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-51165985-87cc-404d-9b2f-dba2bb45bef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:07.945188 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-8e7897f9-d2bd-4788-839a-4041ca97c43c None None] MFA Rules not processed for user `7e8983c2c24941948a58d3c8838e7851`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:07.962899 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8e7897f9-d2bd-4788-839a-4041ca97c43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:08.237999 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51165985-87cc-404d-9b2f-dba2bb45bef3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:08.275338 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 836/1663] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:08.279449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd584d15-8d3f-436d-b836-017d27d10bfc None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:08.282407 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd584d15-8d3f-436d-b836-017d27d10bfc None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:08.283114 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd584d15-8d3f-436d-b836-017d27d10bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:08.380172 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-cd584d15-8d3f-436d-b836-017d27d10bfc None None] MFA Rules not processed for user `640a5815b1a545fc952ac67f553b6797`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:08.385396 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cd584d15-8d3f-436d-b836-017d27d10bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:08.387303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:08.441681 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db282523-2bca-4e89-b710-d7053f345a2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:08.443914 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db282523-2bca-4e89-b710-d7053f345a2a service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:08.444259 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db282523-2bca-4e89-b710-d7053f345a2a service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:08.444869 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db282523-2bca-4e89-b710-d7053f345a2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:08.571277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 837/1664] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:08.573655 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd584d15-8d3f-436d-b836-017d27d10bfc None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:08.610724 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db282523-2bca-4e89-b710-d7053f345a2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:08.612886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db282523-2bca-4e89-b710-d7053f345a2a service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:08.625451 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 828/1665] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 691 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:01:08.627532 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c03334-d70b-461d-ae6b-7da6dd9e0668 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:08.627927 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c03334-d70b-461d-ae6b-7da6dd9e0668 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:08.628299 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c03334-d70b-461d-ae6b-7da6dd9e0668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:08.658252 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-90c03334-d70b-461d-ae6b-7da6dd9e0668 None None] MFA Rules not processed for user `640a5815b1a545fc952ac67f553b6797`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:08.665161 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-90c03334-d70b-461d-ae6b-7da6dd9e0668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:08.691218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 829/1666] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:08.691218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c03334-d70b-461d-ae6b-7da6dd9e0668 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:08.697488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-90c03334-d70b-461d-ae6b-7da6dd9e0668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:08.774123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd584d15-8d3f-436d-b836-017d27d10bfc None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:08.785357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6391ef1-fdbd-4206-b024-7dc8363ece10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:08.786545 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6391ef1-fdbd-4206-b024-7dc8363ece10 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:08.786872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6391ef1-fdbd-4206-b024-7dc8363ece10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:08.787151 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6391ef1-fdbd-4206-b024-7dc8363ece10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:08.787804 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6391ef1-fdbd-4206-b024-7dc8363ece10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:08.790051 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6391ef1-fdbd-4206-b024-7dc8363ece10 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:08.872057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c03334-d70b-461d-ae6b-7da6dd9e0668 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:08.887097 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea3033-3617-45e9-93be-903efeece1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:08.888047 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea3033-3617-45e9-93be-903efeece1e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:08.888142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea3033-3617-45e9-93be-903efeece1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:08.888368 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea3033-3617-45e9-93be-903efeece1e0 admin admin] PATH_INFO: `/v3/projects/41cfe24bec5d4adaa1c42c19634fe164` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:08.902103 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea3033-3617-45e9-93be-903efeece1e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41cfe24bec5d4adaa1c42c19634fe164)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:08.907834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea3033-3617-45e9-93be-903efeece1e0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:08.980616 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 838/1667] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:01:08 2022] POST /identity/v3/projects => generated 375 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:01:08.984588 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6391ef1-fdbd-4206-b024-7dc8363ece10 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:08.990692 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-d6391ef1-fdbd-4206-b024-7dc8363ece10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:09.014958 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:09.086590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d0ea3033-3617-45e9-93be-903efeece1e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41cfe24bec5d4adaa1c42c19634fe164', 'request_id': 'req-d0ea3033-3617-45e9-93be-903efeece1e0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:09.103725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d0ea3033-3617-45e9-93be-903efeece1e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41cfe24bec5d4adaa1c42c19634fe164', 'request_id': 'req-d0ea3033-3617-45e9-93be-903efeece1e0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:09.116887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d0ea3033-3617-45e9-93be-903efeece1e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41cfe24bec5d4adaa1c42c19634fe164', 'request_id': 'req-d0ea3033-3617-45e9-93be-903efeece1e0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:09.173947 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6391ef1-fdbd-4206-b024-7dc8363ece10 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:09.198210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d0ea3033-3617-45e9-93be-903efeece1e0 admin admin] The token cache is being invalidate because project 41cfe24bec5d4adaa1c42c19634fe164 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:01:09.198505 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d0ea3033-3617-45e9-93be-903efeece1e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:09.199271 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d0ea3033-3617-45e9-93be-903efeece1e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:09.201293 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 830/1668] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:01:08 2022] DELETE /identity/v3/projects/41cfe24bec5d4adaa1c42c19634fe164 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:09.202252 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea3033-3617-45e9-93be-903efeece1e0 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:09.206139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d0ea3033-3617-45e9-93be-903efeece1e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:09.230255 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:09.357370 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be22b7b5-6a3c-40d4-bc49-a6d1d820f799 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:09.358003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22b7b5-6a3c-40d4-bc49-a6d1d820f799 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:09.358283 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22b7b5-6a3c-40d4-bc49-a6d1d820f799 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:09.358554 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22b7b5-6a3c-40d4-bc49-a6d1d820f799 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:09.361128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-be22b7b5-6a3c-40d4-bc49-a6d1d820f799 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:09.378354 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea3033-3617-45e9-93be-903efeece1e0 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:09.392947 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e43570-daa1-4b04-8f43-91893212585d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:09.394096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e43570-daa1-4b04-8f43-91893212585d admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:09.394479 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e43570-daa1-4b04-8f43-91893212585d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:09.394882 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e43570-daa1-4b04-8f43-91893212585d admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:09.396064 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e43570-daa1-4b04-8f43-91893212585d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:09.398570 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e43570-daa1-4b04-8f43-91893212585d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:09.494216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 831/1669] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:01:09 2022] POST /identity/v3/users => generated 294 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:01:09.495690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb38e78-ea3b-4d1b-a7a7-f360f50d67d5 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:09.496283 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb38e78-ea3b-4d1b-a7a7-f360f50d67d5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:09.496682 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb38e78-ea3b-4d1b-a7a7-f360f50d67d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:09.531524 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be22b7b5-6a3c-40d4-bc49-a6d1d820f799 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:09.538564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be22b7b5-6a3c-40d4-bc49-a6d1d820f799 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:09.559205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-3fb38e78-ea3b-4d1b-a7a7-f360f50d67d5 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:09.561769 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3fb38e78-ea3b-4d1b-a7a7-f360f50d67d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:09.584770 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 839/1670] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:01:09.587247 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2c538-620f-4457-9c5b-0eec20c33b0d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:09.590732 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 832/1671] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:09.596599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2c538-620f-4457-9c5b-0eec20c33b0d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:09.596599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2c538-620f-4457-9c5b-0eec20c33b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:09.596825 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb38e78-ea3b-4d1b-a7a7-f360f50d67d5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:09.625634 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb38e78-ea3b-4d1b-a7a7-f360f50d67d5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:09.640998 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5405f8c-073f-4c03-9af1-275eb38dcc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:09.642637 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5405f8c-073f-4c03-9af1-275eb38dcc2f admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:09.644980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5405f8c-073f-4c03-9af1-275eb38dcc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:09.645280 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5405f8c-073f-4c03-9af1-275eb38dcc2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:09.645957 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5405f8c-073f-4c03-9af1-275eb38dcc2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:09.650580 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5405f8c-073f-4c03-9af1-275eb38dcc2f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:09.676695 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-60b2c538-620f-4457-9c5b-0eec20c33b0d None None] MFA Rules not processed for user `b8ceeb13c4c24ceb9ff1f27b4e45f62c`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:09.677197 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 833/1672] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:01:09.681034 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d347c-760b-4810-bfe9-c6d55d92ed5f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:09.681322 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d347c-760b-4810-bfe9-c6d55d92ed5f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:09.681601 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d347c-760b-4810-bfe9-c6d55d92ed5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:09.682538 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-60b2c538-620f-4457-9c5b-0eec20c33b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:09.745655 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d63d347c-760b-4810-bfe9-c6d55d92ed5f None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:09.750994 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d63d347c-760b-4810-bfe9-c6d55d92ed5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:09.781813 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 834/1673] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:09.781813 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d347c-760b-4810-bfe9-c6d55d92ed5f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:09.789031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:09.806337 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d347c-760b-4810-bfe9-c6d55d92ed5f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:09.822220 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743c0ea-9f42-466d-abc6-7473aa63ebbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:09.825353 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743c0ea-9f42-466d-abc6-7473aa63ebbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:09.836694 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743c0ea-9f42-466d-abc6-7473aa63ebbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:09.836694 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743c0ea-9f42-466d-abc6-7473aa63ebbf admin admin] PATH_INFO: `/v3/projects/0aa74a0835b44db2856ffb220bc997ea/users/51d6b30cd73742b484134f30a96de4b5/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:09.907370 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743c0ea-9f42-466d-abc6-7473aa63ebbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aa74a0835b44db2856ffb220bc997ea, user_id=51d6b30cd73742b484134f30a96de4b5, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:09.907471 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 840/1674] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:09.907471 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2c538-620f-4457-9c5b-0eec20c33b0d None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:09.909658 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743c0ea-9f42-466d-abc6-7473aa63ebbf admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:09.951068 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2c538-620f-4457-9c5b-0eec20c33b0d None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:09.956983 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bcf999-0668-42c7-8e15-2748677be6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:09.957611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bcf999-0668-42c7-8e15-2748677be6a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:09.958378 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bcf999-0668-42c7-8e15-2748677be6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:09.958378 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bcf999-0668-42c7-8e15-2748677be6a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:09.966472 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bcf999-0668-42c7-8e15-2748677be6a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:01:09.967114 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bcf999-0668-42c7-8e15-2748677be6a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:09.969041 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bcf999-0668-42c7-8e15-2748677be6a9 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:10.009529 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 835/1675] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:01:09 2022] PUT /identity/v3/projects/0aa74a0835b44db2856ffb220bc997ea/users/51d6b30cd73742b484134f30a96de4b5/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:10.011309 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf711d4d-acb3-4345-ae5d-4edfc6825e62 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:10.011604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf711d4d-acb3-4345-ae5d-4edfc6825e62 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:10.011881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf711d4d-acb3-4345-ae5d-4edfc6825e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:10.014629 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 841/1676] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:01:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:01:10.015618 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bcf999-0668-42c7-8e15-2748677be6a9 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:10.048155 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bcf999-0668-42c7-8e15-2748677be6a9 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:10.068944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-cf711d4d-acb3-4345-ae5d-4edfc6825e62 None None] MFA Rules not processed for user `b8ceeb13c4c24ceb9ff1f27b4e45f62c`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:10.070181 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-cf711d4d-acb3-4345-ae5d-4edfc6825e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:10.308139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 836/1677] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:10 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:10.308139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf711d4d-acb3-4345-ae5d-4edfc6825e62 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:10.334552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf711d4d-acb3-4345-ae5d-4edfc6825e62 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:10.338633 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c1da3-067f-4b25-8cf4-ec2ed853cbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:10.339684 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c1da3-067f-4b25-8cf4-ec2ed853cbb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:10.340242 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c1da3-067f-4b25-8cf4-ec2ed853cbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:10.341500 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c1da3-067f-4b25-8cf4-ec2ed853cbb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:10.343735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236c1da3-067f-4b25-8cf4-ec2ed853cbb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:10.352636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236c1da3-067f-4b25-8cf4-ec2ed853cbb2 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:10.356493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d1507-51c7-4289-aced-97aa48e98b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:10.357102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d1507-51c7-4289-aced-97aa48e98b7b admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:10.357381 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d1507-51c7-4289-aced-97aa48e98b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:10.357658 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d1507-51c7-4289-aced-97aa48e98b7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:10.358314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88d1507-51c7-4289-aced-97aa48e98b7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:10.361054 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88d1507-51c7-4289-aced-97aa48e98b7b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:10.374113 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 842/1678] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:01:10 2022] GET /identity/v3/roles => generated 1420 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:01:10.375592 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c1da3-067f-4b25-8cf4-ec2ed853cbb2 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:10.378559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-236c1da3-067f-4b25-8cf4-ec2ed853cbb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:10.459540 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 837/1679] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:01:10 2022] POST /identity/v3/projects => generated 371 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:01:10.466541 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d1507-51c7-4289-aced-97aa48e98b7b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:10.544136 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c1da3-067f-4b25-8cf4-ec2ed853cbb2 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:10.613428 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:10.728055 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d1507-51c7-4289-aced-97aa48e98b7b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:10.763438 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8441fb0a-912a-4305-a78b-11e27d8ce07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:10.764131 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441fb0a-912a-4305-a78b-11e27d8ce07b admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:10.764411 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441fb0a-912a-4305-a78b-11e27d8ce07b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:10.764685 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441fb0a-912a-4305-a78b-11e27d8ce07b admin admin] PATH_INFO: `/v3/projects/0aa74a0835b44db2856ffb220bc997ea/users/51d6b30cd73742b484134f30a96de4b5/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:10.771517 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8441fb0a-912a-4305-a78b-11e27d8ce07b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aa74a0835b44db2856ffb220bc997ea, user_id=51d6b30cd73742b484134f30a96de4b5, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:10.774269 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8441fb0a-912a-4305-a78b-11e27d8ce07b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:10.788102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:10.819199 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1491f-f7a1-4c6b-8c72-51402f7fb08a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:10.819915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1491f-f7a1-4c6b-8c72-51402f7fb08a service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:10.820391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1491f-f7a1-4c6b-8c72-51402f7fb08a service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:10.820742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1491f-f7a1-4c6b-8c72-51402f7fb08a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:10.823548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 838/1680] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:01:10 2022] PUT /identity/v3/projects/0aa74a0835b44db2856ffb220bc997ea/users/51d6b30cd73742b484134f30a96de4b5/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:10.824531 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8441fb0a-912a-4305-a78b-11e27d8ce07b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:10.940973 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1491f-f7a1-4c6b-8c72-51402f7fb08a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:10.940973 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1491f-f7a1-4c6b-8c72-51402f7fb08a service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:10.959806 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 843/1681] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:10 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:01:10.965111 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da2b90-62a4-43a5-a398-a5efdd2a0172 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:10.967198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da2b90-62a4-43a5-a398-a5efdd2a0172 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:10.967498 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da2b90-62a4-43a5-a398-a5efdd2a0172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:11.049880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-70da2b90-62a4-43a5-a398-a5efdd2a0172 None None] MFA Rules not processed for user `a66188ce11bc403d9c657efd78f3019a`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:11.055995 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-70da2b90-62a4-43a5-a398-a5efdd2a0172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:11.066452 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8441fb0a-912a-4305-a78b-11e27d8ce07b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:11.085868 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8554dc1-a20f-4c86-9cab-09f205bebbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:11.086578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8554dc1-a20f-4c86-9cab-09f205bebbd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:11.086931 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8554dc1-a20f-4c86-9cab-09f205bebbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:11.088205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8554dc1-a20f-4c86-9cab-09f205bebbd5 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:11.088205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8554dc1-a20f-4c86-9cab-09f205bebbd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:11.089793 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8554dc1-a20f-4c86-9cab-09f205bebbd5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:11.186611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 844/1682] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:11.188165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee21557-b0fb-4579-9e2a-e3cc1d5aec60 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:11.188509 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee21557-b0fb-4579-9e2a-e3cc1d5aec60 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:11.188791 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee21557-b0fb-4579-9e2a-e3cc1d5aec60 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:11.195516 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 845/1683] 10.222.0.17 () {54 vars in 912 bytes} [Sun Sep 11 00:01:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:01:11.198352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82946a50-e2b0-4e86-b84f-235f9205d601 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:11.198640 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82946a50-e2b0-4e86-b84f-235f9205d601 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:11.198942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82946a50-e2b0-4e86-b84f-235f9205d601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:11.237605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 839/1684] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:01:10 2022] POST /identity/v3/users => generated 286 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:01:11.240888 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6a7a8-60ad-45b3-bc9c-9521cc2586e8 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:11.240888 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6a7a8-60ad-45b3-bc9c-9521cc2586e8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:11.242782 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6a7a8-60ad-45b3-bc9c-9521cc2586e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:11.261869 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-82946a50-e2b0-4e86-b84f-235f9205d601 None None] MFA Rules not processed for user `51d6b30cd73742b484134f30a96de4b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:11.271365 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-82946a50-e2b0-4e86-b84f-235f9205d601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:11.367950 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-24d6a7a8-60ad-45b3-bc9c-9521cc2586e8 None None] MFA Rules not processed for user `53e79d9fb2cd452d8d999a84b6425959`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:11.374506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-24d6a7a8-60ad-45b3-bc9c-9521cc2586e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:11.556265 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 846/1685] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:11.557141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82946a50-e2b0-4e86-b84f-235f9205d601 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:11.586265 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82946a50-e2b0-4e86-b84f-235f9205d601 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:11.607978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f7706-abff-4263-9bad-9f82c5e8122e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:11.608719 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f7706-abff-4263-9bad-9f82c5e8122e admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:11.609003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f7706-abff-4263-9bad-9f82c5e8122e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:11.609274 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f7706-abff-4263-9bad-9f82c5e8122e admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:11.610042 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f7706-abff-4263-9bad-9f82c5e8122e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:11.613401 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f7706-abff-4263-9bad-9f82c5e8122e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:11.615091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:11.649701 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 847/1686] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:01:11.650772 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f7706-abff-4263-9bad-9f82c5e8122e admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:11.658523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-508f7706-abff-4263-9bad-9f82c5e8122e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:11.660476 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 840/1687] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Sep 11 00:01:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 422 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:11.661529 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6a7a8-60ad-45b3-bc9c-9521cc2586e8 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:11.716431 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6a7a8-60ad-45b3-bc9c-9521cc2586e8 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:11.740051 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55664122-22eb-4ec4-8c0f-dc26fd220f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:11.740961 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55664122-22eb-4ec4-8c0f-dc26fd220f3d admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:11.749475 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55664122-22eb-4ec4-8c0f-dc26fd220f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:11.749475 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55664122-22eb-4ec4-8c0f-dc26fd220f3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:11.749475 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55664122-22eb-4ec4-8c0f-dc26fd220f3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:11.758617 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55664122-22eb-4ec4-8c0f-dc26fd220f3d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:11.793336 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 841/1688] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:01:11.807994 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55664122-22eb-4ec4-8c0f-dc26fd220f3d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:11.834685 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55664122-22eb-4ec4-8c0f-dc26fd220f3d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:11.840561 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f7706-abff-4263-9bad-9f82c5e8122e admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:11.842529 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab4cf5-9f1f-45be-8efd-7ad06f8c3c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:11.843507 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab4cf5-9f1f-45be-8efd-7ad06f8c3c3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:11.843795 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab4cf5-9f1f-45be-8efd-7ad06f8c3c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:11.844096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab4cf5-9f1f-45be-8efd-7ad06f8c3c3f admin admin] PATH_INFO: `/v3/projects/2ae4551e32934b4abb156e35c37e054c/users/6246bb42cdea47099d697d94c166a59c/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:11.844500 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb389e1-c5f6-47a3-9e67-d635dc329424 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:11.845068 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb389e1-c5f6-47a3-9e67-d635dc329424 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:11.845336 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb389e1-c5f6-47a3-9e67-d635dc329424 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:11.845611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb389e1-c5f6-47a3-9e67-d635dc329424 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:11.855861 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3eb389e1-c5f6-47a3-9e67-d635dc329424 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:11.893562 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab4cf5-9f1f-45be-8efd-7ad06f8c3c3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ae4551e32934b4abb156e35c37e054c, user_id=6246bb42cdea47099d697d94c166a59c, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:11.896642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab4cf5-9f1f-45be-8efd-7ad06f8c3c3f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:11.995341 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 842/1689] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:01:11 2022] PUT /identity/v3/projects/2ae4551e32934b4abb156e35c37e054c/users/6246bb42cdea47099d697d94c166a59c/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:12.001095 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a9cfc-ea5e-4aa6-9b58-a4e9349d3f9d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:12.001419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a9cfc-ea5e-4aa6-9b58-a4e9349d3f9d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:12.001693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a9cfc-ea5e-4aa6-9b58-a4e9349d3f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:12.018074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb389e1-c5f6-47a3-9e67-d635dc329424 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:12.019582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb389e1-c5f6-47a3-9e67-d635dc329424 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:12.055044 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 848/1690] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:01:12.056281 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb389e1-c5f6-47a3-9e67-d635dc329424 service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:12.059322 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3eb389e1-c5f6-47a3-9e67-d635dc329424 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:12.088169 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a56a9cfc-ea5e-4aa6-9b58-a4e9349d3f9d None None] MFA Rules not processed for user `6246bb42cdea47099d697d94c166a59c`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:12.089386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a56a9cfc-ea5e-4aa6-9b58-a4e9349d3f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:12.094381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb389e1-c5f6-47a3-9e67-d635dc329424 service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:12.101826 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:12.239355 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c3edf-8f89-4067-bf61-db249212eb38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:12.241103 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c3edf-8f89-4067-bf61-db249212eb38 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:12.242096 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c3edf-8f89-4067-bf61-db249212eb38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:12.242492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c3edf-8f89-4067-bf61-db249212eb38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:12.265292 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 843/1691] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:12.265605 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c3edf-8f89-4067-bf61-db249212eb38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:12.265693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6836fea-7eeb-44df-839a-53a7a7d91030 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:12.265693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6836fea-7eeb-44df-839a-53a7a7d91030 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:12.265693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6836fea-7eeb-44df-839a-53a7a7d91030 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:12.265910 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c3edf-8f89-4067-bf61-db249212eb38 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:12.269296 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 844/1692] 10.222.0.17 () {54 vars in 912 bytes} [Sun Sep 11 00:01:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:01:12.270362 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6836fea-7eeb-44df-839a-53a7a7d91030 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:12.315716 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 849/1693] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:01:12.318466 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedd42f-bfb5-492f-bde5-c4c002b7a6a4 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:12.319421 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedd42f-bfb5-492f-bde5-c4c002b7a6a4 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:12.320852 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedd42f-bfb5-492f-bde5-c4c002b7a6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:12.355340 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-3eedd42f-bfb5-492f-bde5-c4c002b7a6a4 None None] MFA Rules not processed for user `53e79d9fb2cd452d8d999a84b6425959`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:12.362045 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3eedd42f-bfb5-492f-bde5-c4c002b7a6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:12.371901 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 850/1694] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Sep 11 00:01:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:12.385254 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eedd42f-bfb5-492f-bde5-c4c002b7a6a4 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:12.387023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3eedd42f-bfb5-492f-bde5-c4c002b7a6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:12.412948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eedd42f-bfb5-492f-bde5-c4c002b7a6a4 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:12.417228 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6836fea-7eeb-44df-839a-53a7a7d91030 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:12.425431 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3a3be-9d31-4285-a9be-08856071edf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:12.425431 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3a3be-9d31-4285-a9be-08856071edf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:12.425431 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3a3be-9d31-4285-a9be-08856071edf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:12.425431 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3a3be-9d31-4285-a9be-08856071edf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:12.426585 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc3a3be-9d31-4285-a9be-08856071edf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:12.428779 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc3a3be-9d31-4285-a9be-08856071edf7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:12.431386 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de4158-8cda-4ede-bc38-d83f38547384 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:12.434267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de4158-8cda-4ede-bc38-d83f38547384 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:12.435209 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de4158-8cda-4ede-bc38-d83f38547384 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:12.436578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de4158-8cda-4ede-bc38-d83f38547384 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:12.466613 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de4158-8cda-4ede-bc38-d83f38547384 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:12.468349 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de4158-8cda-4ede-bc38-d83f38547384 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:12.490217 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 851/1695] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:01:12.536450 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 845/1696] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:01:12 2022] POST /identity/v3/projects => generated 373 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:01:12.540721 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de4158-8cda-4ede-bc38-d83f38547384 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:12.624886 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de4158-8cda-4ede-bc38-d83f38547384 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:12.672797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:12.707282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07addffd-fa41-4d95-8c81-ae3be51b760b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:12.708410 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07addffd-fa41-4d95-8c81-ae3be51b760b admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:12.709077 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07addffd-fa41-4d95-8c81-ae3be51b760b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:12.710194 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07addffd-fa41-4d95-8c81-ae3be51b760b admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:12.711115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07addffd-fa41-4d95-8c81-ae3be51b760b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:12.714365 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07addffd-fa41-4d95-8c81-ae3be51b760b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:12.862291 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 852/1697] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:01:12 2022] POST /identity/v3/users => generated 293 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:01:12.863672 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3a3be-9d31-4285-a9be-08856071edf7 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:12.893555 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3a3be-9d31-4285-a9be-08856071edf7 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:12.898933 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37048c-f6c0-4729-b838-7805ffe86627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:12.900252 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37048c-f6c0-4729-b838-7805ffe86627 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:12.901317 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37048c-f6c0-4729-b838-7805ffe86627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:12.901775 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37048c-f6c0-4729-b838-7805ffe86627 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:12.902599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37048c-f6c0-4729-b838-7805ffe86627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:12.904626 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37048c-f6c0-4729-b838-7805ffe86627 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:12.958583 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 846/1698] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:01:12.978733 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07addffd-fa41-4d95-8c81-ae3be51b760b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:13.019834 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07addffd-fa41-4d95-8c81-ae3be51b760b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:13.040670 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c0b05d-3dd2-4bc8-b13a-44d429e69153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:13.042259 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0b05d-3dd2-4bc8-b13a-44d429e69153 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:13.043413 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0b05d-3dd2-4bc8-b13a-44d429e69153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:13.045085 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0b05d-3dd2-4bc8-b13a-44d429e69153 admin admin] PATH_INFO: `/v3/projects/6616ea8c138946b7b5c37829439bcd6c/users/6d5ba1e5cd29443a85fe7347b9d3fe4c/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:13.073402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c0b05d-3dd2-4bc8-b13a-44d429e69153 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6616ea8c138946b7b5c37829439bcd6c, user_id=6d5ba1e5cd29443a85fe7347b9d3fe4c, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:13.076158 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c0b05d-3dd2-4bc8-b13a-44d429e69153 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:13.226739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 853/1699] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:01:12 2022] PUT /identity/v3/projects/6616ea8c138946b7b5c37829439bcd6c/users/6d5ba1e5cd29443a85fe7347b9d3fe4c/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:13.226987 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37048c-f6c0-4729-b838-7805ffe86627 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:13.240957 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37048c-f6c0-4729-b838-7805ffe86627 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:13.530142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cfbad9-990d-48bc-bef7-6fe00a90ede3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:13.530860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfbad9-990d-48bc-bef7-6fe00a90ede3 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:13.531233 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfbad9-990d-48bc-bef7-6fe00a90ede3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:13.531804 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfbad9-990d-48bc-bef7-6fe00a90ede3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:13.532513 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cfbad9-990d-48bc-bef7-6fe00a90ede3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:13.535548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cfbad9-990d-48bc-bef7-6fe00a90ede3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:13.565360 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 847/1700] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:01:13.565476 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c0b05d-3dd2-4bc8-b13a-44d429e69153 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:13.589849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c0b05d-3dd2-4bc8-b13a-44d429e69153 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:13.602497 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28829d-b0f6-4429-8ffd-f4fa5fc0d9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:13.603244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28829d-b0f6-4429-8ffd-f4fa5fc0d9ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:13.603497 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28829d-b0f6-4429-8ffd-f4fa5fc0d9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:13.603769 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28829d-b0f6-4429-8ffd-f4fa5fc0d9ae admin admin] PATH_INFO: `/v3/projects/6616ea8c138946b7b5c37829439bcd6c/users/6d5ba1e5cd29443a85fe7347b9d3fe4c/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:13.617576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f28829d-b0f6-4429-8ffd-f4fa5fc0d9ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=6616ea8c138946b7b5c37829439bcd6c, user_id=6d5ba1e5cd29443a85fe7347b9d3fe4c, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:13.619652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f28829d-b0f6-4429-8ffd-f4fa5fc0d9ae admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:13.666322 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 854/1701] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:01:13 2022] PUT /identity/v3/projects/6616ea8c138946b7b5c37829439bcd6c/users/6d5ba1e5cd29443a85fe7347b9d3fe4c/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:13.692508 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63799d-dc94-43f3-ae77-4ec091ba7eaf None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:13.693052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63799d-dc94-43f3-ae77-4ec091ba7eaf None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:13.693339 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63799d-dc94-43f3-ae77-4ec091ba7eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:13.717846 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28829d-b0f6-4429-8ffd-f4fa5fc0d9ae admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:13.767931 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-dd63799d-dc94-43f3-ae77-4ec091ba7eaf None None] MFA Rules not processed for user `6d5ba1e5cd29443a85fe7347b9d3fe4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:13.769824 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-dd63799d-dc94-43f3-ae77-4ec091ba7eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:13.774998 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:13.955777 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28829d-b0f6-4429-8ffd-f4fa5fc0d9ae admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:13.966925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1530dd88-fd01-4cf3-9288-70bd2912cfe4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:13.967560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530dd88-fd01-4cf3-9288-70bd2912cfe4 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:13.967903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530dd88-fd01-4cf3-9288-70bd2912cfe4 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:13.968180 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530dd88-fd01-4cf3-9288-70bd2912cfe4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:13.985126 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 848/1702] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:13.985126 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd63799d-dc94-43f3-ae77-4ec091ba7eaf None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:13.985126 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-dd63799d-dc94-43f3-ae77-4ec091ba7eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:14.096977 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1530dd88-fd01-4cf3-9288-70bd2912cfe4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:14.099016 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1530dd88-fd01-4cf3-9288-70bd2912cfe4 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:14.112943 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 855/1703] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:13 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:01:14.115789 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f720f8-0467-41cc-abae-fb4496e38fe6 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:14.116827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f720f8-0467-41cc-abae-fb4496e38fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:14.117104 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd63799d-dc94-43f3-ae77-4ec091ba7eaf None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:14.117600 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f720f8-0467-41cc-abae-fb4496e38fe6 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:14.122163 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 856/1704] 10.222.0.17 () {54 vars in 961 bytes} [Sun Sep 11 00:01:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:01:14.123885 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f720f8-0467-41cc-abae-fb4496e38fe6 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:14.126485 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-63f720f8-0467-41cc-abae-fb4496e38fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:14.131147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a341c6-9724-4b27-a937-e4168459d692 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cb1d85a338e40f18e61887a1becb906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:14.131991 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a341c6-9724-4b27-a937-e4168459d692 service placement] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:14.132421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a341c6-9724-4b27-a937-e4168459d692 service placement] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:14.132813 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a341c6-9724-4b27-a937-e4168459d692 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:14.142748 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d5a341c6-9724-4b27-a937-e4168459d692 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:14.165674 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a341c6-9724-4b27-a937-e4168459d692 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:14.167319 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a341c6-9724-4b27-a937-e4168459d692 service placement] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:14.187105 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 849/1705] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:01:14.190291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592cfa68-0ca3-43fb-b892-faf0fe622f70 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:14.190443 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:14.190768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592cfa68-0ca3-43fb-b892-faf0fe622f70 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:14.190960 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592cfa68-0ca3-43fb-b892-faf0fe622f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:14.239752 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-592cfa68-0ca3-43fb-b892-faf0fe622f70 None None] MFA Rules not processed for user `b8ceeb13c4c24ceb9ff1f27b4e45f62c`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:14.241076 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-592cfa68-0ca3-43fb-b892-faf0fe622f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:14.265514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f720f8-0467-41cc-abae-fb4496e38fe6 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:14.276309 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb2c02-5e06-48f9-b918-1849dc1d3aa0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:14.277202 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb2c02-5e06-48f9-b918-1849dc1d3aa0 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:14.278762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb2c02-5e06-48f9-b918-1849dc1d3aa0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:14.279286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb2c02-5e06-48f9-b918-1849dc1d3aa0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:14.299673 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb2c02-5e06-48f9-b918-1849dc1d3aa0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:14.301582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb2c02-5e06-48f9-b918-1849dc1d3aa0 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:14.319950 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 857/1706] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:01:14.323822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009a037-ad21-4d16-922c-a052c48e277d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:14.325168 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009a037-ad21-4d16-922c-a052c48e277d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:14.325939 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009a037-ad21-4d16-922c-a052c48e277d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:14.386332 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-0009a037-ad21-4d16-922c-a052c48e277d None None] MFA Rules not processed for user `5cb1d85a338e40f18e61887a1becb906`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:14.389335 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0009a037-ad21-4d16-922c-a052c48e277d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:14.409901 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 858/1707] 10.222.0.17 () {58 vars in 1058 bytes} [Sun Sep 11 00:01:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:14.418711 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 850/1708] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:14.418924 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0009a037-ad21-4d16-922c-a052c48e277d None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:14.418924 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0009a037-ad21-4d16-922c-a052c48e277d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:14.420230 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592cfa68-0ca3-43fb-b892-faf0fe622f70 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:14.457302 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592cfa68-0ca3-43fb-b892-faf0fe622f70 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:14.475947 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f5744-af85-41be-9105-fcbc6d7d25f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cb1d85a338e40f18e61887a1becb906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:14.476581 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f5744-af85-41be-9105-fcbc6d7d25f0 service placement] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:14.476859 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f5744-af85-41be-9105-fcbc6d7d25f0 service placement] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:14.479215 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f5744-af85-41be-9105-fcbc6d7d25f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:14.515832 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f5744-af85-41be-9105-fcbc6d7d25f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:14.524671 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f5744-af85-41be-9105-fcbc6d7d25f0 service placement] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:14.573127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 851/1709] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:01:14.622290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0009a037-ad21-4d16-922c-a052c48e277d None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:14.639058 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f016a7e1-78fb-4f74-bad6-0c18d3f1c0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:14.639870 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016a7e1-78fb-4f74-bad6-0c18d3f1c0ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:14.640152 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016a7e1-78fb-4f74-bad6-0c18d3f1c0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:14.640425 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016a7e1-78fb-4f74-bad6-0c18d3f1c0ab admin admin] PATH_INFO: `/v3/users/7a4d4018f1f24a3c9b9172607545ad28` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:14.642225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f016a7e1-78fb-4f74-bad6-0c18d3f1c0ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a4d4018f1f24a3c9b9172607545ad28)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:14.646484 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f016a7e1-78fb-4f74-bad6-0c18d3f1c0ab admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:14.802491 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e4d07-9fbe-4ce7-ae2b-fa4a379fb5f9 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:14.808657 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e4d07-9fbe-4ce7-ae2b-fa4a379fb5f9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:14.808657 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e4d07-9fbe-4ce7-ae2b-fa4a379fb5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:14.862823 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-8e5e4d07-9fbe-4ce7-ae2b-fa4a379fb5f9 None None] MFA Rules not processed for user `9950f578162449c0845c3aa4aab692d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:14.864627 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8e5e4d07-9fbe-4ce7-ae2b-fa4a379fb5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:14.926690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f016a7e1-78fb-4f74-bad6-0c18d3f1c0ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a4d4018f1f24a3c9b9172607545ad28'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:14.950971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:14.969802 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f016a7e1-78fb-4f74-bad6-0c18d3f1c0ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a4d4018f1f24a3c9b9172607545ad28'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:14.978675 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f016a7e1-78fb-4f74-bad6-0c18d3f1c0ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a4d4018f1f24a3c9b9172607545ad28'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:14.981266 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f016a7e1-78fb-4f74-bad6-0c18d3f1c0ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a4d4018f1f24a3c9b9172607545ad28'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:15.035387 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 852/1710] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:15.087678 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f016a7e1-78fb-4f74-bad6-0c18d3f1c0ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a4d4018f1f24a3c9b9172607545ad28'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:15.096340 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 859/1711] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:01:14 2022] DELETE /identity/v3/users/7a4d4018f1f24a3c9b9172607545ad28 => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:15.100460 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c167d5-6412-4660-9881-9ce1719869ee None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:15.100753 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c167d5-6412-4660-9881-9ce1719869ee None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:15.101034 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c167d5-6412-4660-9881-9ce1719869ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:15.128266 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-21c167d5-6412-4660-9881-9ce1719869ee None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:15.128266 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-21c167d5-6412-4660-9881-9ce1719869ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:15.276913 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 853/1712] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:15.290333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f016a7e1-78fb-4f74-bad6-0c18d3f1c0ab admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:15.294795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f016a7e1-78fb-4f74-bad6-0c18d3f1c0ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:15.409467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:15.447090 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f016a7e1-78fb-4f74-bad6-0c18d3f1c0ab admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:15.458699 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773ee56-1031-484c-96e7-b6a51b8f58fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:15.462364 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773ee56-1031-484c-96e7-b6a51b8f58fe service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:15.462364 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773ee56-1031-484c-96e7-b6a51b8f58fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:15.462364 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773ee56-1031-484c-96e7-b6a51b8f58fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:15.478327 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773ee56-1031-484c-96e7-b6a51b8f58fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:15.479823 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773ee56-1031-484c-96e7-b6a51b8f58fe service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:15.508267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 860/1713] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:01:16.103513 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c167d5-6412-4660-9881-9ce1719869ee None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:16.106790 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-21c167d5-6412-4660-9881-9ce1719869ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:16.165415 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c167d5-6412-4660-9881-9ce1719869ee None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:16.175002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce55a85-12b7-405c-abda-6ff2eb72bf76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:16.175680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce55a85-12b7-405c-abda-6ff2eb72bf76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:16.176021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce55a85-12b7-405c-abda-6ff2eb72bf76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:16.176296 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce55a85-12b7-405c-abda-6ff2eb72bf76 admin admin] PATH_INFO: `/v3/users/6246bb42cdea47099d697d94c166a59c` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:16.180100 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce55a85-12b7-405c-abda-6ff2eb72bf76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6246bb42cdea47099d697d94c166a59c)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:16.210885 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce55a85-12b7-405c-abda-6ff2eb72bf76 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:16.626791 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5ce55a85-12b7-405c-abda-6ff2eb72bf76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6246bb42cdea47099d697d94c166a59c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:16.639821 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5ce55a85-12b7-405c-abda-6ff2eb72bf76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6246bb42cdea47099d697d94c166a59c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:16.706880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5ce55a85-12b7-405c-abda-6ff2eb72bf76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6246bb42cdea47099d697d94c166a59c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:16.791144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5ce55a85-12b7-405c-abda-6ff2eb72bf76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6246bb42cdea47099d697d94c166a59c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:16.792452 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5ce55a85-12b7-405c-abda-6ff2eb72bf76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6246bb42cdea47099d697d94c166a59c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:16.799018 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773ee56-1031-484c-96e7-b6a51b8f58fe service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:16.799018 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8773ee56-1031-484c-96e7-b6a51b8f58fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:16.816634 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 854/1714] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:01:16 2022] DELETE /identity/v3/users/6246bb42cdea47099d697d94c166a59c => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:16.822617 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168c044-6965-49f4-bad8-2398eb9caea4 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:16.824421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168c044-6965-49f4-bad8-2398eb9caea4 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:16.827710 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168c044-6965-49f4-bad8-2398eb9caea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:16.836484 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773ee56-1031-484c-96e7-b6a51b8f58fe service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:16.896496 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7168c044-6965-49f4-bad8-2398eb9caea4 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:16.897783 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7168c044-6965-49f4-bad8-2398eb9caea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:17.034203 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c4bad-734f-4797-b735-7851573d37d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:17.036292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c4bad-734f-4797-b735-7851573d37d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:17.037373 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c4bad-734f-4797-b735-7851573d37d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:17.041553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c4bad-734f-4797-b735-7851573d37d8 admin admin] PATH_INFO: `/v3/projects/61a75581f5eb4854959399c3b2d3a3fc` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:17.050768 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c4bad-734f-4797-b735-7851573d37d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61a75581f5eb4854959399c3b2d3a3fc)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:17.065441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c4bad-734f-4797-b735-7851573d37d8 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:17.095217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:17.156040 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 855/1715] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:17.188208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7168c044-6965-49f4-bad8-2398eb9caea4 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:17.192568 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7168c044-6965-49f4-bad8-2398eb9caea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:17.252261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-709c4bad-734f-4797-b735-7851573d37d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61a75581f5eb4854959399c3b2d3a3fc', 'request_id': 'req-709c4bad-734f-4797-b735-7851573d37d8'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:17.330515 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7168c044-6965-49f4-bad8-2398eb9caea4 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:17.343260 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-709c4bad-734f-4797-b735-7851573d37d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61a75581f5eb4854959399c3b2d3a3fc', 'request_id': 'req-709c4bad-734f-4797-b735-7851573d37d8'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:17.350178 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-709c4bad-734f-4797-b735-7851573d37d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61a75581f5eb4854959399c3b2d3a3fc', 'request_id': 'req-709c4bad-734f-4797-b735-7851573d37d8'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:17.367023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-709c4bad-734f-4797-b735-7851573d37d8 admin admin] The token cache is being invalidate because project 61a75581f5eb4854959399c3b2d3a3fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:01:17.367575 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-709c4bad-734f-4797-b735-7851573d37d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:17.368432 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-709c4bad-734f-4797-b735-7851573d37d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:17.373671 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 861/1716] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:01:16 2022] DELETE /identity/v3/projects/61a75581f5eb4854959399c3b2d3a3fc => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:01:17.376033 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e268fcd-7a8e-4b54-ab40-6c5bdef13077 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:17.376555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e268fcd-7a8e-4b54-ab40-6c5bdef13077 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:17.376992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e268fcd-7a8e-4b54-ab40-6c5bdef13077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:17.424821 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-5e268fcd-7a8e-4b54-ab40-6c5bdef13077 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:17.427721 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5e268fcd-7a8e-4b54-ab40-6c5bdef13077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:17.434023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1386ba4-ae83-4677-a6c1-6a5884a20b2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:17.434927 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1386ba4-ae83-4677-a6c1-6a5884a20b2a service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:17.435347 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1386ba4-ae83-4677-a6c1-6a5884a20b2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:17.435766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1386ba4-ae83-4677-a6c1-6a5884a20b2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:17.451428 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e1386ba4-ae83-4677-a6c1-6a5884a20b2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:17.616644 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 862/1717] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:17.624567 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0f978-ae2a-4e0a-ab09-d520210af050 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:17.624847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0f978-ae2a-4e0a-ab09-d520210af050 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:17.625128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0f978-ae2a-4e0a-ab09-d520210af050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:17.663470 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-22c0f978-ae2a-4e0a-ab09-d520210af050 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:17.667972 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-22c0f978-ae2a-4e0a-ab09-d520210af050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:17.668641 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1386ba4-ae83-4677-a6c1-6a5884a20b2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:17.670367 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1386ba4-ae83-4677-a6c1-6a5884a20b2a service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:17.677420 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 863/1718] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:17.678380 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0f978-ae2a-4e0a-ab09-d520210af050 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:17.684483 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:17.691001 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0f978-ae2a-4e0a-ab09-d520210af050 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:17.692914 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 856/1719] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:01:17.723023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656b41d-645c-468d-a5ae-e9e68a873e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:17.736465 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656b41d-645c-468d-a5ae-e9e68a873e73 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:17.743237 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656b41d-645c-468d-a5ae-e9e68a873e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:17.744736 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656b41d-645c-468d-a5ae-e9e68a873e73 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:17.773805 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8656b41d-645c-468d-a5ae-e9e68a873e73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:01:17.773805 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8656b41d-645c-468d-a5ae-e9e68a873e73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:17.773805 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8656b41d-645c-468d-a5ae-e9e68a873e73 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:17.785299 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 864/1720] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:01:17.786520 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1386ba4-ae83-4677-a6c1-6a5884a20b2a service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:17.799458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1386ba4-ae83-4677-a6c1-6a5884a20b2a service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:17.806073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f742a-3538-4134-ad83-d1503fe09f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:17.806740 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f742a-3538-4134-ad83-d1503fe09f55 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:17.806995 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f742a-3538-4134-ad83-d1503fe09f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:17.807520 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f742a-3538-4134-ad83-d1503fe09f55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:17.808163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f742a-3538-4134-ad83-d1503fe09f55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:17.810755 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f742a-3538-4134-ad83-d1503fe09f55 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:17.937147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 857/1721] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:01:17 2022] POST /identity/v3/projects => generated 405 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:01:17.942133 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656b41d-645c-468d-a5ae-e9e68a873e73 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:17.961814 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656b41d-645c-468d-a5ae-e9e68a873e73 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:18.111177 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980eb235-d44f-48ba-a997-ed99c1a7d69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:18.111890 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980eb235-d44f-48ba-a997-ed99c1a7d69c admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:18.112375 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980eb235-d44f-48ba-a997-ed99c1a7d69c admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:18.112652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980eb235-d44f-48ba-a997-ed99c1a7d69c admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:18.113681 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980eb235-d44f-48ba-a997-ed99c1a7d69c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:18.115689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980eb235-d44f-48ba-a997-ed99c1a7d69c admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:18.230045 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 865/1722] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:01:17 2022] POST /identity/v3/users => generated 303 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:01:18.232276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f742a-3538-4134-ad83-d1503fe09f55 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:18.244296 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f742a-3538-4134-ad83-d1503fe09f55 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:18.250368 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe89ca-3213-4f0e-9415-839d01915e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:18.250368 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe89ca-3213-4f0e-9415-839d01915e99 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:18.250368 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe89ca-3213-4f0e-9415-839d01915e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:18.250368 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe89ca-3213-4f0e-9415-839d01915e99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:18.251204 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fe89ca-3213-4f0e-9415-839d01915e99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:18.252894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fe89ca-3213-4f0e-9415-839d01915e99 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:18.269999 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 858/1723] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:01:18.271714 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980eb235-d44f-48ba-a997-ed99c1a7d69c admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:18.291615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980eb235-d44f-48ba-a997-ed99c1a7d69c admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:18.303784 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd9550-6e1d-4c49-a36b-968266211c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:18.304816 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd9550-6e1d-4c49-a36b-968266211c4b admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:18.308470 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd9550-6e1d-4c49-a36b-968266211c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:18.314047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd9550-6e1d-4c49-a36b-968266211c4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:18.314047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dd9550-6e1d-4c49-a36b-968266211c4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:18.314047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dd9550-6e1d-4c49-a36b-968266211c4b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:18.343349 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 866/1724] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:01:18.347983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe89ca-3213-4f0e-9415-839d01915e99 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:18.372602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe89ca-3213-4f0e-9415-839d01915e99 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:18.384098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10b692a-952b-4773-b84d-964eed10cae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:18.384783 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b692a-952b-4773-b84d-964eed10cae8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:18.385057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b692a-952b-4773-b84d-964eed10cae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:18.385325 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b692a-952b-4773-b84d-964eed10cae8 admin admin] PATH_INFO: `/v3/projects/13e027f30c1344c58601199b6e39b174/users/2af1fe7bb6494f8ebd09a66c1ec4138f/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:18.399946 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:18.404711 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10b692a-952b-4773-b84d-964eed10cae8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13e027f30c1344c58601199b6e39b174, user_id=2af1fe7bb6494f8ebd09a66c1ec4138f, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:18.409228 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10b692a-952b-4773-b84d-964eed10cae8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:18.533202 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 859/1725] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:01:18 2022] PUT /identity/v3/projects/13e027f30c1344c58601199b6e39b174/users/2af1fe7bb6494f8ebd09a66c1ec4138f/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:01:18.544312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76e77e-dab8-4675-88f3-3375bee3092a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:18.544759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76e77e-dab8-4675-88f3-3375bee3092a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:18.545133 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76e77e-dab8-4675-88f3-3375bee3092a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:18.627022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-6f76e77e-dab8-4675-88f3-3375bee3092a None None] MFA Rules not processed for user `2af1fe7bb6494f8ebd09a66c1ec4138f`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:18.628805 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6f76e77e-dab8-4675-88f3-3375bee3092a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:18.721822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 867/1726] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:18 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:01:18.726476 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10b692a-952b-4773-b84d-964eed10cae8 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:18.740584 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f76e77e-dab8-4675-88f3-3375bee3092a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:18.746735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6f76e77e-dab8-4675-88f3-3375bee3092a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:18.882601 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10b692a-952b-4773-b84d-964eed10cae8 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:18.898138 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f76e77e-dab8-4675-88f3-3375bee3092a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:18.908200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c33b6f-c727-4e20-aac5-c9637bd5c9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:18.909380 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c33b6f-c727-4e20-aac5-c9637bd5c9ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:18.909892 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc35255-4dae-4564-8a82-bd5bcb49aa68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:18.910103 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c33b6f-c727-4e20-aac5-c9637bd5c9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:18.910531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c33b6f-c727-4e20-aac5-c9637bd5c9ec admin admin] PATH_INFO: `/v3/projects/2ae4551e32934b4abb156e35c37e054c` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:18.910855 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc35255-4dae-4564-8a82-bd5bcb49aa68 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:18.911137 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc35255-4dae-4564-8a82-bd5bcb49aa68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:18.911462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc35255-4dae-4564-8a82-bd5bcb49aa68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:18.912094 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc35255-4dae-4564-8a82-bd5bcb49aa68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:18.912675 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c33b6f-c727-4e20-aac5-c9637bd5c9ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ae4551e32934b4abb156e35c37e054c)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:18.914055 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc35255-4dae-4564-8a82-bd5bcb49aa68 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:18.915313 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c33b6f-c727-4e20-aac5-c9637bd5c9ec admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:18.917664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:19.049110 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 860/1727] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:01:18 2022] POST /identity/v3/projects => generated 403 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:01:19.050350 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc35255-4dae-4564-8a82-bd5bcb49aa68 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:19.080372 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-92c33b6f-c727-4e20-aac5-c9637bd5c9ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ae4551e32934b4abb156e35c37e054c', 'request_id': 'req-92c33b6f-c727-4e20-aac5-c9637bd5c9ec'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:19.206787 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-92c33b6f-c727-4e20-aac5-c9637bd5c9ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ae4551e32934b4abb156e35c37e054c', 'request_id': 'req-92c33b6f-c727-4e20-aac5-c9637bd5c9ec'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:19.214908 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-92c33b6f-c727-4e20-aac5-c9637bd5c9ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ae4551e32934b4abb156e35c37e054c', 'request_id': 'req-92c33b6f-c727-4e20-aac5-c9637bd5c9ec'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:19.215345 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc35255-4dae-4564-8a82-bd5bcb49aa68 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:19.231741 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b127904-ac42-4f99-bec8-4725b74bac49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:19.232538 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b127904-ac42-4f99-bec8-4725b74bac49 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:19.233145 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b127904-ac42-4f99-bec8-4725b74bac49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:19.233855 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b127904-ac42-4f99-bec8-4725b74bac49 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:19.235530 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b127904-ac42-4f99-bec8-4725b74bac49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:19.238107 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b127904-ac42-4f99-bec8-4725b74bac49 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:19.258935 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-92c33b6f-c727-4e20-aac5-c9637bd5c9ec admin admin] The token cache is being invalidate because project 2ae4551e32934b4abb156e35c37e054c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:01:19.259493 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-92c33b6f-c727-4e20-aac5-c9637bd5c9ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:19.261107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-92c33b6f-c727-4e20-aac5-c9637bd5c9ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:19.265725 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 868/1728] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:01:18 2022] DELETE /identity/v3/projects/2ae4551e32934b4abb156e35c37e054c => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:19.267770 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd371b-52f0-4cf2-9d11-d5b5a4498f47 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:19.274229 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd371b-52f0-4cf2-9d11-d5b5a4498f47 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:19.274229 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd371b-52f0-4cf2-9d11-d5b5a4498f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:19.339506 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-08fd371b-52f0-4cf2-9d11-d5b5a4498f47 None None] MFA Rules not processed for user `a66188ce11bc403d9c657efd78f3019a`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:19.355056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-08fd371b-52f0-4cf2-9d11-d5b5a4498f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:19.358140 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 861/1729] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:01:19 2022] POST /identity/v3/users => generated 308 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:01:19.362340 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b127904-ac42-4f99-bec8-4725b74bac49 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:19.383357 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3b127904-ac42-4f99-bec8-4725b74bac49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:19.414144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b127904-ac42-4f99-bec8-4725b74bac49 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:19.426441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e3363-9cde-481e-af84-06081fadbaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:19.427378 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e3363-9cde-481e-af84-06081fadbaed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:19.427652 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e3363-9cde-481e-af84-06081fadbaed admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:19.427920 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e3363-9cde-481e-af84-06081fadbaed admin admin] PATH_INFO: `/v3/users/6d5ba1e5cd29443a85fe7347b9d3fe4c` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:19.432750 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e3363-9cde-481e-af84-06081fadbaed admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d5ba1e5cd29443a85fe7347b9d3fe4c)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:19.435319 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e3363-9cde-481e-af84-06081fadbaed admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:19.486814 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 869/1730] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:19 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:19.486814 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd371b-52f0-4cf2-9d11-d5b5a4498f47 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:19.501017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-08fd371b-52f0-4cf2-9d11-d5b5a4498f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:19.528847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd371b-52f0-4cf2-9d11-d5b5a4498f47 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:19.533716 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e2a20-d893-434e-8903-12417fc27f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:19.534332 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e2a20-d893-434e-8903-12417fc27f10 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:19.534615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e2a20-d893-434e-8903-12417fc27f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:19.534993 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e2a20-d893-434e-8903-12417fc27f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:19.535637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e2a20-d893-434e-8903-12417fc27f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:19.537234 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e2a20-d893-434e-8903-12417fc27f10 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:19.550710 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 870/1731] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:01:19 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:01:19.554880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e2a20-d893-434e-8903-12417fc27f10 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:19.563912 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e2a20-d893-434e-8903-12417fc27f10 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:19.569279 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c34d5-2b9d-4878-be5e-017477709ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:19.570036 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c34d5-2b9d-4878-be5e-017477709ae1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:19.570381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c34d5-2b9d-4878-be5e-017477709ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:19.570776 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c34d5-2b9d-4878-be5e-017477709ae1 admin admin] PATH_INFO: `/v3/projects/7332ac9d5e2549cd95706983b64bfea1/users/58102c475a894b54b77e4d3083fae333/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:19.587546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c34d5-2b9d-4878-be5e-017477709ae1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7332ac9d5e2549cd95706983b64bfea1, user_id=58102c475a894b54b77e4d3083fae333, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:19.590332 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c34d5-2b9d-4878-be5e-017477709ae1 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:19.762048 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7f7e3363-9cde-481e-af84-06081fadbaed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d5ba1e5cd29443a85fe7347b9d3fe4c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:19.778806 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7f7e3363-9cde-481e-af84-06081fadbaed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d5ba1e5cd29443a85fe7347b9d3fe4c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:19.778806 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:19.784530 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7f7e3363-9cde-481e-af84-06081fadbaed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d5ba1e5cd29443a85fe7347b9d3fe4c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:19.806611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 871/1732] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:01:19 2022] PUT /identity/v3/projects/7332ac9d5e2549cd95706983b64bfea1/users/58102c475a894b54b77e4d3083fae333/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:19.815272 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c34d5-2b9d-4878-be5e-017477709ae1 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:19.869500 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7f7e3363-9cde-481e-af84-06081fadbaed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d5ba1e5cd29443a85fe7347b9d3fe4c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:19.870426 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7f7e3363-9cde-481e-af84-06081fadbaed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d5ba1e5cd29443a85fe7347b9d3fe4c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:19.884318 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 862/1733] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:01:19 2022] DELETE /identity/v3/users/6d5ba1e5cd29443a85fe7347b9d3fe4c => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:19.895618 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e3363-9cde-481e-af84-06081fadbaed admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:19.897132 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7f7e3363-9cde-481e-af84-06081fadbaed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:19.919472 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c34d5-2b9d-4878-be5e-017477709ae1 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:19.919472 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:19.993134 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e3363-9cde-481e-af84-06081fadbaed admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:19.993134 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e78d3ab-20a1-4a70-b963-af2caae5092e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:19.993134 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78d3ab-20a1-4a70-b963-af2caae5092e service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:19.993134 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78d3ab-20a1-4a70-b963-af2caae5092e service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:19.993673 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78d3ab-20a1-4a70-b963-af2caae5092e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:19.994683 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3e78d3ab-20a1-4a70-b963-af2caae5092e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:19.995148 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65d95e-899a-4ce1-b44b-06e75440ec85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:19.995813 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65d95e-899a-4ce1-b44b-06e75440ec85 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:19.996711 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65d95e-899a-4ce1-b44b-06e75440ec85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:19.996991 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65d95e-899a-4ce1-b44b-06e75440ec85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:19.997691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b65d95e-899a-4ce1-b44b-06e75440ec85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:19.999475 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b65d95e-899a-4ce1-b44b-06e75440ec85 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:20.012496 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 872/1734] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:01:19 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:01:20.017639 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65d95e-899a-4ce1-b44b-06e75440ec85 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:20.019213 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2b65d95e-899a-4ce1-b44b-06e75440ec85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:20.035639 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65d95e-899a-4ce1-b44b-06e75440ec85 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:20.041415 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5ee8f-cb85-4eea-b004-7ec9c9dd649c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:20.042081 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5ee8f-cb85-4eea-b004-7ec9c9dd649c admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:20.042455 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5ee8f-cb85-4eea-b004-7ec9c9dd649c admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:20.042756 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5ee8f-cb85-4eea-b004-7ec9c9dd649c admin admin] PATH_INFO: `/v3/projects/7332ac9d5e2549cd95706983b64bfea1/users/58102c475a894b54b77e4d3083fae333/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:20.044771 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5ee8f-cb85-4eea-b004-7ec9c9dd649c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7332ac9d5e2549cd95706983b64bfea1, user_id=58102c475a894b54b77e4d3083fae333, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:20.048092 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5ee8f-cb85-4eea-b004-7ec9c9dd649c admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:20.059123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 873/1735] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:01:20 2022] PUT /identity/v3/projects/7332ac9d5e2549cd95706983b64bfea1/users/58102c475a894b54b77e4d3083fae333/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:01:20.070143 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5a83c-1291-4d4f-a42b-0db34de78eaa None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:20.070907 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5a83c-1291-4d4f-a42b-0db34de78eaa None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:20.071243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5a83c-1291-4d4f-a42b-0db34de78eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:20.077736 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e78d3ab-20a1-4a70-b963-af2caae5092e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:20.080009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e78d3ab-20a1-4a70-b963-af2caae5092e service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:20.091114 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 863/1736] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:19 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:01:20.106220 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-8dd5a83c-1291-4d4f-a42b-0db34de78eaa None None] MFA Rules not processed for user `58102c475a894b54b77e4d3083fae333`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:20.107764 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8dd5a83c-1291-4d4f-a42b-0db34de78eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:20.184470 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 874/1737] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:20 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:01:20.196175 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e78d3ab-20a1-4a70-b963-af2caae5092e service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:20.197618 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3e78d3ab-20a1-4a70-b963-af2caae5092e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:20.210022 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e78d3ab-20a1-4a70-b963-af2caae5092e service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:20.296589 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a0720b-b66a-4f60-a04c-f80c7ca53ce2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:20.297791 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0720b-b66a-4f60-a04c-f80c7ca53ce2 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:20.298705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0720b-b66a-4f60-a04c-f80c7ca53ce2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:20.299001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0720b-b66a-4f60-a04c-f80c7ca53ce2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:20.312424 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a0720b-b66a-4f60-a04c-f80c7ca53ce2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:20.318205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a0720b-b66a-4f60-a04c-f80c7ca53ce2 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:20.333175 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 864/1738] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:20 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:01:20.602826 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5a83c-1291-4d4f-a42b-0db34de78eaa None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:20.607484 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8dd5a83c-1291-4d4f-a42b-0db34de78eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:20.756835 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a0720b-b66a-4f60-a04c-f80c7ca53ce2 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:20.885334 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5a83c-1291-4d4f-a42b-0db34de78eaa None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:20.887699 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:20.897364 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460853ea-afa3-4b10-9f6a-13c29169f433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:20.900080 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460853ea-afa3-4b10-9f6a-13c29169f433 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:20.900642 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460853ea-afa3-4b10-9f6a-13c29169f433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:20.901221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460853ea-afa3-4b10-9f6a-13c29169f433 admin admin] PATH_INFO: `/v3/projects/6616ea8c138946b7b5c37829439bcd6c` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:20.903600 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460853ea-afa3-4b10-9f6a-13c29169f433 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6616ea8c138946b7b5c37829439bcd6c)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:20.906532 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460853ea-afa3-4b10-9f6a-13c29169f433 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:20.974904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a0720b-b66a-4f60-a04c-f80c7ca53ce2 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:20.987339 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8b871-430e-4868-a69f-f1458dc31e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:20.988007 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8b871-430e-4868-a69f-f1458dc31e82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:20.988285 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8b871-430e-4868-a69f-f1458dc31e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:20.988557 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8b871-430e-4868-a69f-f1458dc31e82 admin admin] PATH_INFO: `/v3/users/2af1fe7bb6494f8ebd09a66c1ec4138f` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:20.990402 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8b871-430e-4868-a69f-f1458dc31e82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2af1fe7bb6494f8ebd09a66c1ec4138f)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:20.992924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8b871-430e-4868-a69f-f1458dc31e82 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:20.993979 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:21.150179 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-460853ea-afa3-4b10-9f6a-13c29169f433 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6616ea8c138946b7b5c37829439bcd6c', 'request_id': 'req-460853ea-afa3-4b10-9f6a-13c29169f433'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:21.248408 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-460853ea-afa3-4b10-9f6a-13c29169f433 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6616ea8c138946b7b5c37829439bcd6c', 'request_id': 'req-460853ea-afa3-4b10-9f6a-13c29169f433'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:21.260552 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-460853ea-afa3-4b10-9f6a-13c29169f433 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6616ea8c138946b7b5c37829439bcd6c', 'request_id': 'req-460853ea-afa3-4b10-9f6a-13c29169f433'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:21.271138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fca8b871-430e-4868-a69f-f1458dc31e82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2af1fe7bb6494f8ebd09a66c1ec4138f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:21.277199 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-460853ea-afa3-4b10-9f6a-13c29169f433 admin admin] The token cache is being invalidate because project 6616ea8c138946b7b5c37829439bcd6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:01:21.282024 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-460853ea-afa3-4b10-9f6a-13c29169f433 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:21.282024 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-460853ea-afa3-4b10-9f6a-13c29169f433 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:21.282024 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 875/1739] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:01:20 2022] DELETE /identity/v3/projects/6616ea8c138946b7b5c37829439bcd6c => generated 0 bytes in 678 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:21.283469 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fca8b871-430e-4868-a69f-f1458dc31e82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2af1fe7bb6494f8ebd09a66c1ec4138f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:21.286036 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ddbe3-93d8-4ef0-8ca3-6373407e3c2f None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:21.286472 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ddbe3-93d8-4ef0-8ca3-6373407e3c2f None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:21.287893 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ddbe3-93d8-4ef0-8ca3-6373407e3c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:21.311124 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fca8b871-430e-4868-a69f-f1458dc31e82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2af1fe7bb6494f8ebd09a66c1ec4138f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:21.321820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-cd1ddbe3-93d8-4ef0-8ca3-6373407e3c2f None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:21.326538 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cd1ddbe3-93d8-4ef0-8ca3-6373407e3c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:21.411362 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fca8b871-430e-4868-a69f-f1458dc31e82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2af1fe7bb6494f8ebd09a66c1ec4138f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:21.411991 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fca8b871-430e-4868-a69f-f1458dc31e82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2af1fe7bb6494f8ebd09a66c1ec4138f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:21.418685 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 865/1740] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:01:20 2022] DELETE /identity/v3/users/2af1fe7bb6494f8ebd09a66c1ec4138f => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:21.423627 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f182587-1099-4971-82f5-b2f618c827cf None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:21.423919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f182587-1099-4971-82f5-b2f618c827cf None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:21.424193 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f182587-1099-4971-82f5-b2f618c827cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:21.452897 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 876/1741] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:01:21.463123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a86b78-68f6-49e0-8f69-7fd5d5c6eadd None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:21.463123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a86b78-68f6-49e0-8f69-7fd5d5c6eadd None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:21.463452 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a86b78-68f6-49e0-8f69-7fd5d5c6eadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:21.470911 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-3f182587-1099-4971-82f5-b2f618c827cf None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:21.472657 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3f182587-1099-4971-82f5-b2f618c827cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:21.501267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-96a86b78-68f6-49e0-8f69-7fd5d5c6eadd None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:21.503088 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-96a86b78-68f6-49e0-8f69-7fd5d5c6eadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:21.622060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 877/1742] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:01:21.627432 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a86b78-68f6-49e0-8f69-7fd5d5c6eadd None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:21.649211 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a86b78-68f6-49e0-8f69-7fd5d5c6eadd None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:21.661232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb6034-96d8-43c8-b27d-53ada6ec9f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:21.662945 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 866/1743] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:01:21.667646 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb6034-96d8-43c8-b27d-53ada6ec9f45 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:21.668589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb6034-96d8-43c8-b27d-53ada6ec9f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:21.668752 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb6034-96d8-43c8-b27d-53ada6ec9f45 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:21.670874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb6034-96d8-43c8-b27d-53ada6ec9f45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:01:21.670874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb6034-96d8-43c8-b27d-53ada6ec9f45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:21.682045 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb6034-96d8-43c8-b27d-53ada6ec9f45 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:21.691288 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f182587-1099-4971-82f5-b2f618c827cf None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:21.696908 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3f182587-1099-4971-82f5-b2f618c827cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:21.707981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 878/1744] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:01:21.711195 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb6034-96d8-43c8-b27d-53ada6ec9f45 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:21.804992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb6034-96d8-43c8-b27d-53ada6ec9f45 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:21.813546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33568ce0-a271-4254-a055-e600891eca97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:21.814476 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33568ce0-a271-4254-a055-e600891eca97 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:21.814999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33568ce0-a271-4254-a055-e600891eca97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:21.815478 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33568ce0-a271-4254-a055-e600891eca97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:21.816731 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33568ce0-a271-4254-a055-e600891eca97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:21.825190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33568ce0-a271-4254-a055-e600891eca97 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:21.862494 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:21.868907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f182587-1099-4971-82f5-b2f618c827cf None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:21.874561 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c2c68-c299-422b-af62-5665376ff081 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:21.876168 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c2c68-c299-422b-af62-5665376ff081 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:21.876859 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c2c68-c299-422b-af62-5665376ff081 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:21.877368 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c2c68-c299-422b-af62-5665376ff081 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:21.890477 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c2c68-c299-422b-af62-5665376ff081 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:21.893239 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c2c68-c299-422b-af62-5665376ff081 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:21.907873 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 867/1745] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:01:21.923113 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 879/1746] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:01:21 2022] POST /identity/v3/projects => generated 357 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:01:21.924826 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c2c68-c299-422b-af62-5665376ff081 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:21.935971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c2c68-c299-422b-af62-5665376ff081 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:22.041566 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f81ac8-aeac-4586-b031-2bfa0db8c1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:22.042103 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f81ac8-aeac-4586-b031-2bfa0db8c1cb admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:22.042511 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f81ac8-aeac-4586-b031-2bfa0db8c1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:22.042824 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f81ac8-aeac-4586-b031-2bfa0db8c1cb admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:22.043778 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f81ac8-aeac-4586-b031-2bfa0db8c1cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:22.047882 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f81ac8-aeac-4586-b031-2bfa0db8c1cb admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:22.204862 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 868/1747] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:01:21 2022] POST /identity/v3/users => generated 279 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:01:22.209355 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33568ce0-a271-4254-a055-e600891eca97 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:22.221417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33568ce0-a271-4254-a055-e600891eca97 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:22.231520 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165ae7c-5c8a-4445-a4cd-2ac87350f129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:22.232324 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165ae7c-5c8a-4445-a4cd-2ac87350f129 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:22.232608 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165ae7c-5c8a-4445-a4cd-2ac87350f129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:22.232881 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165ae7c-5c8a-4445-a4cd-2ac87350f129 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:22.233567 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8165ae7c-5c8a-4445-a4cd-2ac87350f129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:22.236212 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8165ae7c-5c8a-4445-a4cd-2ac87350f129 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:22.242686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:22.251432 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 880/1748] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:01:22.259814 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f81ac8-aeac-4586-b031-2bfa0db8c1cb admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:22.275433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f81ac8-aeac-4586-b031-2bfa0db8c1cb admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:22.284088 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab30b1e-5f2f-4117-a41b-58acbb718dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:22.285485 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab30b1e-5f2f-4117-a41b-58acbb718dd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:22.285995 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab30b1e-5f2f-4117-a41b-58acbb718dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:22.287796 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab30b1e-5f2f-4117-a41b-58acbb718dd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:22.289150 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab30b1e-5f2f-4117-a41b-58acbb718dd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:22.291945 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab30b1e-5f2f-4117-a41b-58acbb718dd8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:22.353391 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 869/1749] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:01:22.355757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165ae7c-5c8a-4445-a4cd-2ac87350f129 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:22.374057 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165ae7c-5c8a-4445-a4cd-2ac87350f129 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:22.383192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301606a-2ef4-4f89-9f5b-14eaad371654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:22.383997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301606a-2ef4-4f89-9f5b-14eaad371654 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:22.384376 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301606a-2ef4-4f89-9f5b-14eaad371654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:22.384645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301606a-2ef4-4f89-9f5b-14eaad371654 admin admin] PATH_INFO: `/v3/projects/4cd2dbd4a1b442eb86a3d663a9bfd1c1/users/6abd75ccff2241c3bf344c4ce9b34e55/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:22.406009 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a301606a-2ef4-4f89-9f5b-14eaad371654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd2dbd4a1b442eb86a3d663a9bfd1c1, user_id=6abd75ccff2241c3bf344c4ce9b34e55, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:22.409962 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a301606a-2ef4-4f89-9f5b-14eaad371654 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:22.474689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 881/1750] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:01:22 2022] PUT /identity/v3/projects/4cd2dbd4a1b442eb86a3d663a9bfd1c1/users/6abd75ccff2241c3bf344c4ce9b34e55/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:01:22.493345 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb29ab4-4664-4c4d-8e95-f805dae90f93 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:22.493658 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb29ab4-4664-4c4d-8e95-f805dae90f93 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:22.498796 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb29ab4-4664-4c4d-8e95-f805dae90f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:22.544300 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-3cb29ab4-4664-4c4d-8e95-f805dae90f93 None None] MFA Rules not processed for user `6abd75ccff2241c3bf344c4ce9b34e55`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:22.548109 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3cb29ab4-4664-4c4d-8e95-f805dae90f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:22.588130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301606a-2ef4-4f89-9f5b-14eaad371654 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:22.589474 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a301606a-2ef4-4f89-9f5b-14eaad371654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:22.674680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 870/1751] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:22 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:22.688424 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb29ab4-4664-4c4d-8e95-f805dae90f93 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:22.767747 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301606a-2ef4-4f89-9f5b-14eaad371654 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:22.774016 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02641f1f-53e7-4f86-b7c8-06fc74d4a32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:22.774915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02641f1f-53e7-4f86-b7c8-06fc74d4a32b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:22.775190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02641f1f-53e7-4f86-b7c8-06fc74d4a32b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:22.775461 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02641f1f-53e7-4f86-b7c8-06fc74d4a32b admin admin] PATH_INFO: `/v3/projects/13e027f30c1344c58601199b6e39b174` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:22.776627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02641f1f-53e7-4f86-b7c8-06fc74d4a32b admin admin] RBAC: Authorizing `identity:delete_project(project_id=13e027f30c1344c58601199b6e39b174)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:22.778745 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02641f1f-53e7-4f86-b7c8-06fc74d4a32b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:22.818479 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb29ab4-4664-4c4d-8e95-f805dae90f93 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:22.827858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad266e7c-2774-429b-8913-e85cb2bc862a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:22.829185 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad266e7c-2774-429b-8913-e85cb2bc862a admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:22.829616 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad266e7c-2774-429b-8913-e85cb2bc862a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:22.829957 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad266e7c-2774-429b-8913-e85cb2bc862a admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:22.830762 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad266e7c-2774-429b-8913-e85cb2bc862a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:22.834283 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad266e7c-2774-429b-8913-e85cb2bc862a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:22.840283 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:22.930788 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-02641f1f-53e7-4f86-b7c8-06fc74d4a32b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13e027f30c1344c58601199b6e39b174', 'request_id': 'req-02641f1f-53e7-4f86-b7c8-06fc74d4a32b'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:22.981349 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 871/1752] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:01:22 2022] POST /identity/v3/projects => generated 357 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:01:22.983118 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981def1-661e-4e53-b9f2-b0ad69fb9fce None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:22.983411 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981def1-661e-4e53-b9f2-b0ad69fb9fce None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:22.983715 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981def1-661e-4e53-b9f2-b0ad69fb9fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:22.993542 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-02641f1f-53e7-4f86-b7c8-06fc74d4a32b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13e027f30c1344c58601199b6e39b174', 'request_id': 'req-02641f1f-53e7-4f86-b7c8-06fc74d4a32b'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:23.009962 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-02641f1f-53e7-4f86-b7c8-06fc74d4a32b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13e027f30c1344c58601199b6e39b174', 'request_id': 'req-02641f1f-53e7-4f86-b7c8-06fc74d4a32b'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:23.023751 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-02641f1f-53e7-4f86-b7c8-06fc74d4a32b admin admin] The token cache is being invalidate because project 13e027f30c1344c58601199b6e39b174 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:01:23.024070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-02641f1f-53e7-4f86-b7c8-06fc74d4a32b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:23.024731 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-02641f1f-53e7-4f86-b7c8-06fc74d4a32b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:23.026399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 882/1753] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:01:22 2022] DELETE /identity/v3/projects/13e027f30c1344c58601199b6e39b174 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:23.027195 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02641f1f-53e7-4f86-b7c8-06fc74d4a32b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:23.028520 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-02641f1f-53e7-4f86-b7c8-06fc74d4a32b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:23.053155 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7981def1-661e-4e53-b9f2-b0ad69fb9fce None None] MFA Rules not processed for user `b0a879085a194b0687e1cdcbafd48033`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:23.054408 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7981def1-661e-4e53-b9f2-b0ad69fb9fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:23.105986 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02641f1f-53e7-4f86-b7c8-06fc74d4a32b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:23.123305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:23.134265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 872/1754] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Sep 11 00:01:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:23.135965 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981def1-661e-4e53-b9f2-b0ad69fb9fce None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:23.137846 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7981def1-661e-4e53-b9f2-b0ad69fb9fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:23.210287 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50f251-66d2-4b58-9353-f0a640680154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:23.210965 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50f251-66d2-4b58-9353-f0a640680154 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:23.214920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50f251-66d2-4b58-9353-f0a640680154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:23.214920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50f251-66d2-4b58-9353-f0a640680154 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:23.214920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff50f251-66d2-4b58-9353-f0a640680154 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:23.218004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff50f251-66d2-4b58-9353-f0a640680154 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:23.245232 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981def1-661e-4e53-b9f2-b0ad69fb9fce None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:23.251461 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3745d-e9a3-4e93-a666-685a1db94cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:23.252255 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3745d-e9a3-4e93-a666-685a1db94cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:23.252531 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3745d-e9a3-4e93-a666-685a1db94cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:23.252800 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3745d-e9a3-4e93-a666-685a1db94cf5 admin admin] PATH_INFO: `/v3/users/58102c475a894b54b77e4d3083fae333` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:23.254546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3745d-e9a3-4e93-a666-685a1db94cf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58102c475a894b54b77e4d3083fae333)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:23.259353 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3745d-e9a3-4e93-a666-685a1db94cf5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:23.362773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 883/1755] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:01:23 2022] POST /identity/v3/users => generated 285 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:01:23.364167 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50f251-66d2-4b58-9353-f0a640680154 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:23.374239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50f251-66d2-4b58-9353-f0a640680154 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:23.379115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d19760-e1bc-479a-bd3a-8d02c657e041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:23.380096 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d19760-e1bc-479a-bd3a-8d02c657e041 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:23.380096 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d19760-e1bc-479a-bd3a-8d02c657e041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:23.380384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d19760-e1bc-479a-bd3a-8d02c657e041 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:23.380884 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d19760-e1bc-479a-bd3a-8d02c657e041 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:23.382442 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d19760-e1bc-479a-bd3a-8d02c657e041 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:23.392762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 884/1756] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:01:23.396782 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d19760-e1bc-479a-bd3a-8d02c657e041 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:23.406229 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d19760-e1bc-479a-bd3a-8d02c657e041 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:23.411490 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095e0c3-7867-4ad7-a122-cb4ff4582aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:23.412128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095e0c3-7867-4ad7-a122-cb4ff4582aaa admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:23.412408 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095e0c3-7867-4ad7-a122-cb4ff4582aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:23.412681 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095e0c3-7867-4ad7-a122-cb4ff4582aaa admin admin] PATH_INFO: `/v3/projects/8375da4aa42444cabc5b0511a06632ee/users/62d3f21c326e4ba39b165d74e8feb5d3/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:23.427663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e095e0c3-7867-4ad7-a122-cb4ff4582aaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=8375da4aa42444cabc5b0511a06632ee, user_id=62d3f21c326e4ba39b165d74e8feb5d3, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:23.430023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e095e0c3-7867-4ad7-a122-cb4ff4582aaa admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:23.466815 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-66d3745d-e9a3-4e93-a666-685a1db94cf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58102c475a894b54b77e4d3083fae333'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:23.475491 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-66d3745d-e9a3-4e93-a666-685a1db94cf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58102c475a894b54b77e4d3083fae333'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:23.477462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:23.491738 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-66d3745d-e9a3-4e93-a666-685a1db94cf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58102c475a894b54b77e4d3083fae333'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:23.501833 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 885/1757] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:01:23 2022] PUT /identity/v3/projects/8375da4aa42444cabc5b0511a06632ee/users/62d3f21c326e4ba39b165d74e8feb5d3/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:01:23.501833 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095e0c3-7867-4ad7-a122-cb4ff4582aaa admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:23.504050 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e095e0c3-7867-4ad7-a122-cb4ff4582aaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:23.595248 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-66d3745d-e9a3-4e93-a666-685a1db94cf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58102c475a894b54b77e4d3083fae333'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:23.596163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-66d3745d-e9a3-4e93-a666-685a1db94cf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58102c475a894b54b77e4d3083fae333'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:23.598905 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095e0c3-7867-4ad7-a122-cb4ff4582aaa admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:23.603714 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 873/1758] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:01:23 2022] DELETE /identity/v3/users/58102c475a894b54b77e4d3083fae333 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:23.603833 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3745d-e9a3-4e93-a666-685a1db94cf5 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:23.605157 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-66d3745d-e9a3-4e93-a666-685a1db94cf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:23.607309 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14625a92-7de6-471c-970d-ade353c49279 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '564e65585e9543e286b6041af870cf41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:23.607932 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14625a92-7de6-471c-970d-ade353c49279 service swift] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:23.608206 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14625a92-7de6-471c-970d-ade353c49279 service swift] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:23.608471 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14625a92-7de6-471c-970d-ade353c49279 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:23.610523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-14625a92-7de6-471c-970d-ade353c49279 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:23.699093 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14625a92-7de6-471c-970d-ade353c49279 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:23.699712 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14625a92-7de6-471c-970d-ade353c49279 service swift] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:23.722173 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 886/1759] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Sep 11 00:01:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 00:01:23.739290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:23.745493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3745d-e9a3-4e93-a666-685a1db94cf5 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:23.751093 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49f9f1-d487-4170-808b-93362da79b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:23.751712 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49f9f1-d487-4170-808b-93362da79b1d admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:23.751985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49f9f1-d487-4170-808b-93362da79b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:23.752254 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49f9f1-d487-4170-808b-93362da79b1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:23.752894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49f9f1-d487-4170-808b-93362da79b1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:23.754490 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49f9f1-d487-4170-808b-93362da79b1d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:23.765581 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 874/1760] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:01:23.770436 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14625a92-7de6-471c-970d-ade353c49279 service swift] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:23.917764 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14625a92-7de6-471c-970d-ade353c49279 service swift] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:23.928647 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ae419-ec26-49d5-8c6a-2e1f993edca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:23.929608 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ae419-ec26-49d5-8c6a-2e1f993edca8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:23.929894 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ae419-ec26-49d5-8c6a-2e1f993edca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:23.930226 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ae419-ec26-49d5-8c6a-2e1f993edca8 admin admin] PATH_INFO: `/v3/projects/8375da4aa42444cabc5b0511a06632ee/users/62d3f21c326e4ba39b165d74e8feb5d3/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:23.936075 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ae419-ec26-49d5-8c6a-2e1f993edca8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8375da4aa42444cabc5b0511a06632ee, user_id=62d3f21c326e4ba39b165d74e8feb5d3, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:23.942377 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ae419-ec26-49d5-8c6a-2e1f993edca8 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:23.977779 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 887/1761] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:01:23 2022] PUT /identity/v3/projects/8375da4aa42444cabc5b0511a06632ee/users/62d3f21c326e4ba39b165d74e8feb5d3/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:23.989809 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6d913-fecc-48ea-9618-d98ac43be95f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:23.990186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6d913-fecc-48ea-9618-d98ac43be95f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:23.990462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6d913-fecc-48ea-9618-d98ac43be95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:24.038036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-98d6d913-fecc-48ea-9618-d98ac43be95f None None] MFA Rules not processed for user `62d3f21c326e4ba39b165d74e8feb5d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:24.040438 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-98d6d913-fecc-48ea-9618-d98ac43be95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:24.160179 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 875/1762] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:23 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:24.167739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ae419-ec26-49d5-8c6a-2e1f993edca8 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:24.169764 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c25ae419-ec26-49d5-8c6a-2e1f993edca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:24.254353 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d6d913-fecc-48ea-9618-d98ac43be95f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:24.255932 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-98d6d913-fecc-48ea-9618-d98ac43be95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:24.256057 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ae419-ec26-49d5-8c6a-2e1f993edca8 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:24.259799 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31666c-a4a0-4548-9704-8d134a395906 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:24.260548 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31666c-a4a0-4548-9704-8d134a395906 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:24.260825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31666c-a4a0-4548-9704-8d134a395906 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:24.261093 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31666c-a4a0-4548-9704-8d134a395906 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:24.278623 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c31666c-a4a0-4548-9704-8d134a395906 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:24.280603 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c31666c-a4a0-4548-9704-8d134a395906 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:24.309236 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 888/1763] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:24 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:01:24.373864 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:24.379713 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d6d913-fecc-48ea-9618-d98ac43be95f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:24.390694 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc864b89-e609-447c-a8af-c0227fe940fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:24.391548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc864b89-e609-447c-a8af-c0227fe940fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:24.391920 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc864b89-e609-447c-a8af-c0227fe940fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:24.392217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc864b89-e609-447c-a8af-c0227fe940fd admin admin] PATH_INFO: `/v3/projects/7332ac9d5e2549cd95706983b64bfea1` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:24.393401 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc864b89-e609-447c-a8af-c0227fe940fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7332ac9d5e2549cd95706983b64bfea1)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:24.396423 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc864b89-e609-447c-a8af-c0227fe940fd admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:24.590542 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dc864b89-e609-447c-a8af-c0227fe940fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7332ac9d5e2549cd95706983b64bfea1', 'request_id': 'req-dc864b89-e609-447c-a8af-c0227fe940fd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:24.613503 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dc864b89-e609-447c-a8af-c0227fe940fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7332ac9d5e2549cd95706983b64bfea1', 'request_id': 'req-dc864b89-e609-447c-a8af-c0227fe940fd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:24.620361 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dc864b89-e609-447c-a8af-c0227fe940fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7332ac9d5e2549cd95706983b64bfea1', 'request_id': 'req-dc864b89-e609-447c-a8af-c0227fe940fd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:24.632403 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31666c-a4a0-4548-9704-8d134a395906 service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:24.643920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31666c-a4a0-4548-9704-8d134a395906 service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:24.714078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dc864b89-e609-447c-a8af-c0227fe940fd admin admin] The token cache is being invalidate because project 7332ac9d5e2549cd95706983b64bfea1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:01:24.714705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dc864b89-e609-447c-a8af-c0227fe940fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:24.721674 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dc864b89-e609-447c-a8af-c0227fe940fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:24.727408 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 876/1764] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:01:24 2022] DELETE /identity/v3/projects/7332ac9d5e2549cd95706983b64bfea1 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:01:24.731673 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7a1c8-7792-4910-acae-ce47e4945e8b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:24.732253 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7a1c8-7792-4910-acae-ce47e4945e8b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:24.732803 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7a1c8-7792-4910-acae-ce47e4945e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:24.774640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c8d7a1c8-7792-4910-acae-ce47e4945e8b None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:24.776244 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c8d7a1c8-7792-4910-acae-ce47e4945e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:24.784626 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bdc40-d99a-422b-ace7-eb656c44f804 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:24.789233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bdc40-d99a-422b-ace7-eb656c44f804 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:24.789233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bdc40-d99a-422b-ace7-eb656c44f804 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:24.789233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bdc40-d99a-422b-ace7-eb656c44f804 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:24.790389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-481bdc40-d99a-422b-ace7-eb656c44f804 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:24.887585 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 877/1765] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:01:24.899857 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea337e2b-90d7-4048-9243-4cef8c19eb83 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:24.899857 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea337e2b-90d7-4048-9243-4cef8c19eb83 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:24.899857 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea337e2b-90d7-4048-9243-4cef8c19eb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:24.931281 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ea337e2b-90d7-4048-9243-4cef8c19eb83 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:24.932767 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:24.932908 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ea337e2b-90d7-4048-9243-4cef8c19eb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:24.946170 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 878/1766] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:01:24.950000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea337e2b-90d7-4048-9243-4cef8c19eb83 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:24.966106 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea337e2b-90d7-4048-9243-4cef8c19eb83 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:24.972415 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff460b8-4947-457d-8ffc-dcb8ec096c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:24.973136 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff460b8-4947-457d-8ffc-dcb8ec096c8f admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:24.973433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff460b8-4947-457d-8ffc-dcb8ec096c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:24.973714 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff460b8-4947-457d-8ffc-dcb8ec096c8f admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:24.974474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff460b8-4947-457d-8ffc-dcb8ec096c8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:01:24.975022 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff460b8-4947-457d-8ffc-dcb8ec096c8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:24.978012 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff460b8-4947-457d-8ffc-dcb8ec096c8f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:24.982958 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481bdc40-d99a-422b-ace7-eb656c44f804 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:24.982958 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481bdc40-d99a-422b-ace7-eb656c44f804 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:24.999605 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 889/1767] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:24 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:01:25.029985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 879/1768] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:01:25.035980 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bdc40-d99a-422b-ace7-eb656c44f804 service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:25.066746 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bdc40-d99a-422b-ace7-eb656c44f804 service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:25.087048 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66850d4-d3ef-4457-8efc-6b7302fe633c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:25.087048 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66850d4-d3ef-4457-8efc-6b7302fe633c admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:25.087048 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66850d4-d3ef-4457-8efc-6b7302fe633c admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:25.087048 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66850d4-d3ef-4457-8efc-6b7302fe633c admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:25.088025 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66850d4-d3ef-4457-8efc-6b7302fe633c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:25.088749 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcabb1bb-4852-43ea-b526-16862e52722a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:25.089977 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcabb1bb-4852-43ea-b526-16862e52722a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:25.093084 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcabb1bb-4852-43ea-b526-16862e52722a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:25.093535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66850d4-d3ef-4457-8efc-6b7302fe633c admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:25.131144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-dcabb1bb-4852-43ea-b526-16862e52722a None None] MFA Rules not processed for user `53e79d9fb2cd452d8d999a84b6425959`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:25.140019 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-dcabb1bb-4852-43ea-b526-16862e52722a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:25.232283 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 890/1769] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:01:25 2022] POST /identity/v3/projects => generated 383 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:01:25.236697 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66850d4-d3ef-4457-8efc-6b7302fe633c admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:25.372222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 880/1770] 10.222.0.43 () {58 vars in 1015 bytes} [Sun Sep 11 00:01:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:25.410790 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcabb1bb-4852-43ea-b526-16862e52722a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:25.412928 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-dcabb1bb-4852-43ea-b526-16862e52722a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:25.447726 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcabb1bb-4852-43ea-b526-16862e52722a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:25.483051 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66850d4-d3ef-4457-8efc-6b7302fe633c admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:25.500658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0da50-b8ca-49e3-ba06-d56b1c9521e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:25.504581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0da50-b8ca-49e3-ba06-d56b1c9521e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:25.504872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0da50-b8ca-49e3-ba06-d56b1c9521e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:25.505156 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0da50-b8ca-49e3-ba06-d56b1c9521e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:25.505822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e0da50-b8ca-49e3-ba06-d56b1c9521e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:25.520631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e0da50-b8ca-49e3-ba06-d56b1c9521e9 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:25.536720 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:25.655150 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 891/1771] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:01:25 2022] POST /identity/v3/users => generated 292 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:01:25.656600 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0da50-b8ca-49e3-ba06-d56b1c9521e9 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:25.668542 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4d7df-b487-459a-ad89-9908686bbde0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:25.669403 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4d7df-b487-459a-ad89-9908686bbde0 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:25.669780 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4d7df-b487-459a-ad89-9908686bbde0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:25.670154 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4d7df-b487-459a-ad89-9908686bbde0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:25.680828 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e4d7df-b487-459a-ad89-9908686bbde0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:25.682567 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e4d7df-b487-459a-ad89-9908686bbde0 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:25.689301 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0da50-b8ca-49e3-ba06-d56b1c9521e9 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:25.703180 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dce014-7df9-4b5b-97fc-0847fd6fac55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:25.704929 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dce014-7df9-4b5b-97fc-0847fd6fac55 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:25.705475 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dce014-7df9-4b5b-97fc-0847fd6fac55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:25.706162 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dce014-7df9-4b5b-97fc-0847fd6fac55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:25.707078 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dce014-7df9-4b5b-97fc-0847fd6fac55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:25.709231 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dce014-7df9-4b5b-97fc-0847fd6fac55 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:25.730996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 881/1772] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:01:25.732236 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4d7df-b487-459a-ad89-9908686bbde0 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:25.746534 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 892/1773] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:01:25.787931 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4d7df-b487-459a-ad89-9908686bbde0 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:25.821397 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d512567-fa77-40ac-bd4a-add589543ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:25.822725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d512567-fa77-40ac-bd4a-add589543ea8 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:25.827130 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d512567-fa77-40ac-bd4a-add589543ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:25.827782 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d512567-fa77-40ac-bd4a-add589543ea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:25.831076 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d512567-fa77-40ac-bd4a-add589543ea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:25.834447 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d512567-fa77-40ac-bd4a-add589543ea8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:25.901810 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 882/1774] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:01:25.907185 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dce014-7df9-4b5b-97fc-0847fd6fac55 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:25.919040 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dce014-7df9-4b5b-97fc-0847fd6fac55 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:25.927887 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e9d0a-09a5-4892-94c8-47d2105424a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:25.927887 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e9d0a-09a5-4892-94c8-47d2105424a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:25.927887 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e9d0a-09a5-4892-94c8-47d2105424a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:25.927887 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e9d0a-09a5-4892-94c8-47d2105424a6 admin admin] PATH_INFO: `/v3/projects/f76355067c114e43aa1766109a8dcd7b/users/2e4783e836b848918251790549424351/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:25.956691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0e9d0a-09a5-4892-94c8-47d2105424a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f76355067c114e43aa1766109a8dcd7b, user_id=2e4783e836b848918251790549424351, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:25.960474 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0e9d0a-09a5-4892-94c8-47d2105424a6 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:26.088261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 893/1775] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:01:25 2022] PUT /identity/v3/projects/f76355067c114e43aa1766109a8dcd7b/users/2e4783e836b848918251790549424351/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:26.103689 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c0cf1-9688-471a-9a7b-ab7e4da88e3a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:26.104139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c0cf1-9688-471a-9a7b-ab7e4da88e3a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:26.104258 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c0cf1-9688-471a-9a7b-ab7e4da88e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:26.174358 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-287c0cf1-9688-471a-9a7b-ab7e4da88e3a None None] MFA Rules not processed for user `2e4783e836b848918251790549424351`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:26.175665 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-287c0cf1-9688-471a-9a7b-ab7e4da88e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:26.281256 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 883/1776] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:26.286402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e9d0a-09a5-4892-94c8-47d2105424a6 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:26.395234 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e9d0a-09a5-4892-94c8-47d2105424a6 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:26.403070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60617631-eff5-4844-853c-2f6d720b2100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:26.404117 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60617631-eff5-4844-853c-2f6d720b2100 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:26.404509 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60617631-eff5-4844-853c-2f6d720b2100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:26.404886 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60617631-eff5-4844-853c-2f6d720b2100 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:26.405868 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60617631-eff5-4844-853c-2f6d720b2100 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:26.408830 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60617631-eff5-4844-853c-2f6d720b2100 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:26.504150 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:26.527679 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 894/1777] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:01:26 2022] POST /identity/v3/projects => generated 381 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:01:26.528482 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c0cf1-9688-471a-9a7b-ab7e4da88e3a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:26.544907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c0cf1-9688-471a-9a7b-ab7e4da88e3a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:26.691530 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2b371-e6ef-46e1-b974-bf9514651c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:26.693912 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2b371-e6ef-46e1-b974-bf9514651c4a admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:26.694181 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2b371-e6ef-46e1-b974-bf9514651c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:26.694449 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2b371-e6ef-46e1-b974-bf9514651c4a admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:26.695164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2b371-e6ef-46e1-b974-bf9514651c4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:26.697940 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2b371-e6ef-46e1-b974-bf9514651c4a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:26.778736 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 884/1778] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:01:26 2022] POST /identity/v3/users => generated 297 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:01:26.786238 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60617631-eff5-4844-853c-2f6d720b2100 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:26.802810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60617631-eff5-4844-853c-2f6d720b2100 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:26.812395 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92073802-4f0b-4219-8897-d652cccebaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:26.813670 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92073802-4f0b-4219-8897-d652cccebaea admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:26.814154 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92073802-4f0b-4219-8897-d652cccebaea admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:26.814613 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92073802-4f0b-4219-8897-d652cccebaea admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:26.815517 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92073802-4f0b-4219-8897-d652cccebaea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:26.818237 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92073802-4f0b-4219-8897-d652cccebaea admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:26.841800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 895/1779] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:01:26.843160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2b371-e6ef-46e1-b974-bf9514651c4a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:26.872933 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2b371-e6ef-46e1-b974-bf9514651c4a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:26.878688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3772cebf-5e5f-4db5-9ffc-fc28be8d7b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:26.923283 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772cebf-5e5f-4db5-9ffc-fc28be8d7b5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:26.923589 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772cebf-5e5f-4db5-9ffc-fc28be8d7b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:26.929246 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772cebf-5e5f-4db5-9ffc-fc28be8d7b5b admin admin] PATH_INFO: `/v3/projects/03e519c191014e9baf73d613eac8a086/users/2a8b3a9f54064cd4aa6723e79e8b289d/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:27.000923 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3772cebf-5e5f-4db5-9ffc-fc28be8d7b5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=03e519c191014e9baf73d613eac8a086, user_id=2a8b3a9f54064cd4aa6723e79e8b289d, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:27.019044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3772cebf-5e5f-4db5-9ffc-fc28be8d7b5b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:27.152751 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 885/1780] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:01:26 2022] PUT /identity/v3/projects/03e519c191014e9baf73d613eac8a086/users/2a8b3a9f54064cd4aa6723e79e8b289d/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:27.152913 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92073802-4f0b-4219-8897-d652cccebaea admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:27.171005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92073802-4f0b-4219-8897-d652cccebaea admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:27.380159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2615b4-fcc6-452d-86d3-2e4add839f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:27.384891 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2615b4-fcc6-452d-86d3-2e4add839f1a admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:27.385176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2615b4-fcc6-452d-86d3-2e4add839f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:27.385451 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2615b4-fcc6-452d-86d3-2e4add839f1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:27.387638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2615b4-fcc6-452d-86d3-2e4add839f1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:27.398081 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2615b4-fcc6-452d-86d3-2e4add839f1a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:27.424188 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 896/1781] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:01:27.428568 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3772cebf-5e5f-4db5-9ffc-fc28be8d7b5b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:27.446385 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3772cebf-5e5f-4db5-9ffc-fc28be8d7b5b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:27.449096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:27.468214 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b162a4a-ac4d-4427-916c-63db24926736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:27.470044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b162a4a-ac4d-4427-916c-63db24926736 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:27.470579 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b162a4a-ac4d-4427-916c-63db24926736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:27.471384 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b162a4a-ac4d-4427-916c-63db24926736 admin admin] PATH_INFO: `/v3/projects/03e519c191014e9baf73d613eac8a086/users/2a8b3a9f54064cd4aa6723e79e8b289d/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:27.475629 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b162a4a-ac4d-4427-916c-63db24926736 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03e519c191014e9baf73d613eac8a086, user_id=2a8b3a9f54064cd4aa6723e79e8b289d, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:27.480562 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b162a4a-ac4d-4427-916c-63db24926736 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:27.495156 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 886/1782] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:01:27 2022] PUT /identity/v3/projects/03e519c191014e9baf73d613eac8a086/users/2a8b3a9f54064cd4aa6723e79e8b289d/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:27.506621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22107dc5-3509-4be0-9564-9c60436e6fc3 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:27.506621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22107dc5-3509-4be0-9564-9c60436e6fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:27.506621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22107dc5-3509-4be0-9564-9c60436e6fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:27.607917 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-22107dc5-3509-4be0-9564-9c60436e6fc3 None None] MFA Rules not processed for user `2a8b3a9f54064cd4aa6723e79e8b289d`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:27.607917 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-22107dc5-3509-4be0-9564-9c60436e6fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:27.759299 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 897/1783] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:27.782707 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b162a4a-ac4d-4427-916c-63db24926736 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:27.785053 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0b162a4a-ac4d-4427-916c-63db24926736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:27.933472 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b162a4a-ac4d-4427-916c-63db24926736 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:27.946776 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db268d7-0900-4bd2-b1cb-e7efea556662 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:27.950599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db268d7-0900-4bd2-b1cb-e7efea556662 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:27.950599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db268d7-0900-4bd2-b1cb-e7efea556662 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:27.950599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db268d7-0900-4bd2-b1cb-e7efea556662 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:27.963131 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db268d7-0900-4bd2-b1cb-e7efea556662 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:27.964606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db268d7-0900-4bd2-b1cb-e7efea556662 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:27.983641 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 887/1784] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:27 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:01:28.169345 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81114f-eb01-4e13-89cc-3ae67841aa2d None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:28.169345 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81114f-eb01-4e13-89cc-3ae67841aa2d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:28.169345 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81114f-eb01-4e13-89cc-3ae67841aa2d None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:28.175576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 898/1785] 10.222.0.17 () {54 vars in 921 bytes} [Sun Sep 11 00:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:01:28.361884 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d361e8f-c8d0-4a45-8bab-cac6cc3854cb None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:28.362495 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d361e8f-c8d0-4a45-8bab-cac6cc3854cb None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:28.362848 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d361e8f-c8d0-4a45-8bab-cac6cc3854cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:28.376240 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81114f-eb01-4e13-89cc-3ae67841aa2d None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:28.420496 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81114f-eb01-4e13-89cc-3ae67841aa2d None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:28.426494 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b505f-0d5e-4445-bf03-9a096259f68b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:28.429182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b505f-0d5e-4445-bf03-9a096259f68b service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:28.429605 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b505f-0d5e-4445-bf03-9a096259f68b service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:28.429982 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b505f-0d5e-4445-bf03-9a096259f68b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:28.488205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-3d361e8f-c8d0-4a45-8bab-cac6cc3854cb None None] MFA Rules not processed for user `636bfafb693044d283530063047fc0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:28.489744 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3d361e8f-c8d0-4a45-8bab-cac6cc3854cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:28.657870 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91b505f-0d5e-4445-bf03-9a096259f68b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:28.660422 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91b505f-0d5e-4445-bf03-9a096259f68b service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:28.682989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 899/1786] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:01:28.806189 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 888/1787] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Sep 11 00:01:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:28.843324 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b505f-0d5e-4445-bf03-9a096259f68b service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:28.846115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f91b505f-0d5e-4445-bf03-9a096259f68b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:28.886888 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b505f-0d5e-4445-bf03-9a096259f68b service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:28.904815 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300826c5-9531-40f3-aac8-ce1336ffba5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:28.905700 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300826c5-9531-40f3-aac8-ce1336ffba5c service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:28.906520 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300826c5-9531-40f3-aac8-ce1336ffba5c service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:28.908402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300826c5-9531-40f3-aac8-ce1336ffba5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:28.944916 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:28.953850 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300826c5-9531-40f3-aac8-ce1336ffba5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:28.955690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300826c5-9531-40f3-aac8-ce1336ffba5c service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:28.984376 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 900/1788] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:01:30.075668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b6841-75aa-4b0c-ad95-c070890f8254 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:30.076669 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b6841-75aa-4b0c-ad95-c070890f8254 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:30.076945 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b6841-75aa-4b0c-ad95-c070890f8254 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:30.080972 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 889/1789] 10.222.0.17 () {54 vars in 921 bytes} [Sun Sep 11 00:01:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:01:30.105144 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975a8c7-436f-4b14-a2cc-4e40b0c4df5f None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:30.105664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975a8c7-436f-4b14-a2cc-4e40b0c4df5f None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:30.105745 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975a8c7-436f-4b14-a2cc-4e40b0c4df5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:30.129205 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-3975a8c7-436f-4b14-a2cc-4e40b0c4df5f None None] MFA Rules not processed for user `636bfafb693044d283530063047fc0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:30.129205 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3975a8c7-436f-4b14-a2cc-4e40b0c4df5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:30.137348 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 901/1790] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Sep 11 00:01:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:30.149742 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b6841-75aa-4b0c-ad95-c070890f8254 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:30.151622 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-815b6841-75aa-4b0c-ad95-c070890f8254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:30.173223 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b6841-75aa-4b0c-ad95-c070890f8254 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:30.179344 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563861db-d7ae-446e-9f6f-ee07872cfeda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:30.180209 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563861db-d7ae-446e-9f6f-ee07872cfeda service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:30.180534 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563861db-d7ae-446e-9f6f-ee07872cfeda service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:30.181018 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563861db-d7ae-446e-9f6f-ee07872cfeda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:30.191981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563861db-d7ae-446e-9f6f-ee07872cfeda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:30.193416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563861db-d7ae-446e-9f6f-ee07872cfeda service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:30.206945 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 890/1791] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:01:35.939466 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975a8c7-436f-4b14-a2cc-4e40b0c4df5f None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:36.075381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975a8c7-436f-4b14-a2cc-4e40b0c4df5f None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:36.075381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0579ee42-3891-40f3-857e-500560ef99ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:36.075381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579ee42-3891-40f3-857e-500560ef99ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:36.075381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579ee42-3891-40f3-857e-500560ef99ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:36.076380 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579ee42-3891-40f3-857e-500560ef99ee admin admin] PATH_INFO: `/v3/users/2e4783e836b848918251790549424351` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:36.076380 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0579ee42-3891-40f3-857e-500560ef99ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e4783e836b848918251790549424351)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:36.076380 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0579ee42-3891-40f3-857e-500560ef99ee admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:36.414416 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0579ee42-3891-40f3-857e-500560ef99ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e4783e836b848918251790549424351'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:36.439828 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0579ee42-3891-40f3-857e-500560ef99ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e4783e836b848918251790549424351'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:36.459058 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0579ee42-3891-40f3-857e-500560ef99ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e4783e836b848918251790549424351'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:36.469719 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0579ee42-3891-40f3-857e-500560ef99ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e4783e836b848918251790549424351'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:36.640182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0579ee42-3891-40f3-857e-500560ef99ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e4783e836b848918251790549424351'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:36.649848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 902/1792] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:01:35 2022] DELETE /identity/v3/users/2e4783e836b848918251790549424351 => generated 0 bytes in 711 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:36.654613 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575cddc-5c0e-47d5-9bea-2aa594916893 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:36.655257 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575cddc-5c0e-47d5-9bea-2aa594916893 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:36.655257 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575cddc-5c0e-47d5-9bea-2aa594916893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:36.683813 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-6575cddc-5c0e-47d5-9bea-2aa594916893 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:36.685173 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6575cddc-5c0e-47d5-9bea-2aa594916893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:36.787907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 891/1793] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:36.808436 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0579ee42-3891-40f3-857e-500560ef99ee admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:36.809909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0579ee42-3891-40f3-857e-500560ef99ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:36.911956 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0579ee42-3891-40f3-857e-500560ef99ee admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:36.916492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c9e40-de3d-4378-a1b3-e60b722bc7e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:36.917297 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c9e40-de3d-4378-a1b3-e60b722bc7e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:36.917883 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c9e40-de3d-4378-a1b3-e60b722bc7e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:36.918469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c9e40-de3d-4378-a1b3-e60b722bc7e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:36.930849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2c9e40-de3d-4378-a1b3-e60b722bc7e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:36.932759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2c9e40-de3d-4378-a1b3-e60b722bc7e1 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:36.945756 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 903/1794] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:01:37.441371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575cddc-5c0e-47d5-9bea-2aa594916893 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:37.443197 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6575cddc-5c0e-47d5-9bea-2aa594916893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:37.458401 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575cddc-5c0e-47d5-9bea-2aa594916893 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:37.463237 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663557a-ab3e-4ae1-8ae4-4c9c15eba2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:37.464433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663557a-ab3e-4ae1-8ae4-4c9c15eba2ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:37.464880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663557a-ab3e-4ae1-8ae4-4c9c15eba2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:37.465162 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663557a-ab3e-4ae1-8ae4-4c9c15eba2ca admin admin] PATH_INFO: `/v3/projects/f76355067c114e43aa1766109a8dcd7b` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:37.466283 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663557a-ab3e-4ae1-8ae4-4c9c15eba2ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=f76355067c114e43aa1766109a8dcd7b)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:37.468241 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663557a-ab3e-4ae1-8ae4-4c9c15eba2ca admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:37.477062 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:37.566303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b663557a-ab3e-4ae1-8ae4-4c9c15eba2ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f76355067c114e43aa1766109a8dcd7b', 'request_id': 'req-b663557a-ab3e-4ae1-8ae4-4c9c15eba2ca'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:37.572648 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b663557a-ab3e-4ae1-8ae4-4c9c15eba2ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f76355067c114e43aa1766109a8dcd7b', 'request_id': 'req-b663557a-ab3e-4ae1-8ae4-4c9c15eba2ca'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:37.578188 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b663557a-ab3e-4ae1-8ae4-4c9c15eba2ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f76355067c114e43aa1766109a8dcd7b', 'request_id': 'req-b663557a-ab3e-4ae1-8ae4-4c9c15eba2ca'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:37.633556 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b663557a-ab3e-4ae1-8ae4-4c9c15eba2ca admin admin] The token cache is being invalidate because project f76355067c114e43aa1766109a8dcd7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:01:37.633838 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b663557a-ab3e-4ae1-8ae4-4c9c15eba2ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:37.634479 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b663557a-ab3e-4ae1-8ae4-4c9c15eba2ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:37.635987 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 892/1795] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:01:37 2022] DELETE /identity/v3/projects/f76355067c114e43aa1766109a8dcd7b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:37.639930 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c9e40-de3d-4378-a1b3-e60b722bc7e1 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:37.650025 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c9e40-de3d-4378-a1b3-e60b722bc7e1 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:37.654514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:37.705452 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cb51b-9663-439c-bb0e-7c13d8ebe9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:37.706068 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cb51b-9663-439c-bb0e-7c13d8ebe9fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:37.706342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cb51b-9663-439c-bb0e-7c13d8ebe9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:37.706634 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cb51b-9663-439c-bb0e-7c13d8ebe9fa admin admin] PATH_INFO: `/v3/users/2a8b3a9f54064cd4aa6723e79e8b289d` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:37.707880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559cb51b-9663-439c-bb0e-7c13d8ebe9fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a8b3a9f54064cd4aa6723e79e8b289d)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:37.710121 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559cb51b-9663-439c-bb0e-7c13d8ebe9fa admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:37.894359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-559cb51b-9663-439c-bb0e-7c13d8ebe9fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a8b3a9f54064cd4aa6723e79e8b289d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:37.911102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-559cb51b-9663-439c-bb0e-7c13d8ebe9fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a8b3a9f54064cd4aa6723e79e8b289d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:37.919034 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-559cb51b-9663-439c-bb0e-7c13d8ebe9fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a8b3a9f54064cd4aa6723e79e8b289d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:37.919938 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-559cb51b-9663-439c-bb0e-7c13d8ebe9fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a8b3a9f54064cd4aa6723e79e8b289d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:37.994047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-559cb51b-9663-439c-bb0e-7c13d8ebe9fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a8b3a9f54064cd4aa6723e79e8b289d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:38.001206 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 904/1796] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:01:37 2022] DELETE /identity/v3/users/2a8b3a9f54064cd4aa6723e79e8b289d => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:38.463979 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663557a-ab3e-4ae1-8ae4-4c9c15eba2ca admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:38.465535 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b663557a-ab3e-4ae1-8ae4-4c9c15eba2ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:38.478844 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663557a-ab3e-4ae1-8ae4-4c9c15eba2ca admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:38.560639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a9b43d-79f7-42d0-8ff0-92fb1f8955f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:38.564828 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9b43d-79f7-42d0-8ff0-92fb1f8955f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:38.564828 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9b43d-79f7-42d0-8ff0-92fb1f8955f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:38.564828 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9b43d-79f7-42d0-8ff0-92fb1f8955f7 admin admin] PATH_INFO: `/v3/projects/03e519c191014e9baf73d613eac8a086` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:38.564828 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a9b43d-79f7-42d0-8ff0-92fb1f8955f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03e519c191014e9baf73d613eac8a086)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:38.566199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a9b43d-79f7-42d0-8ff0-92fb1f8955f7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:38.723683 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e7a9b43d-79f7-42d0-8ff0-92fb1f8955f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03e519c191014e9baf73d613eac8a086', 'request_id': 'req-e7a9b43d-79f7-42d0-8ff0-92fb1f8955f7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:38.736113 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e7a9b43d-79f7-42d0-8ff0-92fb1f8955f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03e519c191014e9baf73d613eac8a086', 'request_id': 'req-e7a9b43d-79f7-42d0-8ff0-92fb1f8955f7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:38.742857 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e7a9b43d-79f7-42d0-8ff0-92fb1f8955f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03e519c191014e9baf73d613eac8a086', 'request_id': 'req-e7a9b43d-79f7-42d0-8ff0-92fb1f8955f7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:38.811650 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e7a9b43d-79f7-42d0-8ff0-92fb1f8955f7 admin admin] The token cache is being invalidate because project 03e519c191014e9baf73d613eac8a086 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:01:38.811947 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e7a9b43d-79f7-42d0-8ff0-92fb1f8955f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:38.812618 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-e7a9b43d-79f7-42d0-8ff0-92fb1f8955f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:38.828169 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 893/1797] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:01:38 2022] DELETE /identity/v3/projects/03e519c191014e9baf73d613eac8a086 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:38.835353 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91446d9c-e7ed-48b4-b9e7-fd3dc24e0980 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:38.835790 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91446d9c-e7ed-48b4-b9e7-fd3dc24e0980 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:38.835790 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91446d9c-e7ed-48b4-b9e7-fd3dc24e0980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:38.858527 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-91446d9c-e7ed-48b4-b9e7-fd3dc24e0980 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:38.860867 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-91446d9c-e7ed-48b4-b9e7-fd3dc24e0980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:38.931000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 905/1798] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:38.941234 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cfa1fa-e5e9-45a9-808f-2fde141fdeb6 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:38.941513 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cfa1fa-e5e9-45a9-808f-2fde141fdeb6 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:38.941784 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cfa1fa-e5e9-45a9-808f-2fde141fdeb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:38.963857 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-07cfa1fa-e5e9-45a9-808f-2fde141fdeb6 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:38.964716 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-07cfa1fa-e5e9-45a9-808f-2fde141fdeb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:38.973924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 894/1799] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:38.978967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91446d9c-e7ed-48b4-b9e7-fd3dc24e0980 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:38.998480 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91446d9c-e7ed-48b4-b9e7-fd3dc24e0980 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:39.009241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd4bdb-09cd-493a-877b-a3c38a42e598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:39.010010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd4bdb-09cd-493a-877b-a3c38a42e598 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:39.010744 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd4bdb-09cd-493a-877b-a3c38a42e598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:39.011117 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd4bdb-09cd-493a-877b-a3c38a42e598 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:39.011983 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebd4bdb-09cd-493a-877b-a3c38a42e598 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:01:39.012492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebd4bdb-09cd-493a-877b-a3c38a42e598 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:39.014586 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebd4bdb-09cd-493a-877b-a3c38a42e598 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:39.030241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 906/1800] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:01:39.036352 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cfa1fa-e5e9-45a9-808f-2fde141fdeb6 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:39.046239 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cfa1fa-e5e9-45a9-808f-2fde141fdeb6 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:39.051538 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9cf0b-9ca8-4631-b763-86a9aa23d4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:39.052155 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9cf0b-9ca8-4631-b763-86a9aa23d4bb admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:39.052432 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9cf0b-9ca8-4631-b763-86a9aa23d4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:39.052704 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9cf0b-9ca8-4631-b763-86a9aa23d4bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:39.053338 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9cf0b-9ca8-4631-b763-86a9aa23d4bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:39.055320 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9cf0b-9ca8-4631-b763-86a9aa23d4bb admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:39.163933 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:39.175985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 895/1801] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:01:39 2022] POST /identity/v3/projects => generated 407 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:01:39.186249 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd4bdb-09cd-493a-877b-a3c38a42e598 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:39.208371 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd4bdb-09cd-493a-877b-a3c38a42e598 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:39.229179 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:39.290508 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0237d-b73e-4d86-ba43-c8baf6f87de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:39.292059 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0237d-b73e-4d86-ba43-c8baf6f87de1 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:39.292485 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0237d-b73e-4d86-ba43-c8baf6f87de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:39.292608 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0237d-b73e-4d86-ba43-c8baf6f87de1 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:39.293246 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df0237d-b73e-4d86-ba43-c8baf6f87de1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:39.297080 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df0237d-b73e-4d86-ba43-c8baf6f87de1 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:39.404851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 907/1802] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:01:39 2022] POST /identity/v3/users => generated 304 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:01:39.407318 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9cf0b-9ca8-4631-b763-86a9aa23d4bb admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:39.422295 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9cf0b-9ca8-4631-b763-86a9aa23d4bb admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:39.436058 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14b796-e1cb-4bdf-8a7d-26beb88b014e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:39.436847 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14b796-e1cb-4bdf-8a7d-26beb88b014e admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:39.438983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14b796-e1cb-4bdf-8a7d-26beb88b014e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:39.439503 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14b796-e1cb-4bdf-8a7d-26beb88b014e admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:39.440185 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14b796-e1cb-4bdf-8a7d-26beb88b014e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:39.441829 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14b796-e1cb-4bdf-8a7d-26beb88b014e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:39.452806 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 896/1803] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:01:39.458354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0237d-b73e-4d86-ba43-c8baf6f87de1 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:39.470467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0237d-b73e-4d86-ba43-c8baf6f87de1 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:39.477323 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1c5e7-2b4c-4081-98bb-fb18b009c4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:39.477973 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1c5e7-2b4c-4081-98bb-fb18b009c4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:39.478226 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1c5e7-2b4c-4081-98bb-fb18b009c4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:39.478498 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1c5e7-2b4c-4081-98bb-fb18b009c4e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:39.479177 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e1c5e7-2b4c-4081-98bb-fb18b009c4e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:39.481131 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e1c5e7-2b4c-4081-98bb-fb18b009c4e7 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:39.494637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 908/1804] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:01:39.507090 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14b796-e1cb-4bdf-8a7d-26beb88b014e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:39.519290 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14b796-e1cb-4bdf-8a7d-26beb88b014e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:39.525819 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f497e01c-ce7a-4a69-9523-5b8ed6b5813b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:39.526762 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497e01c-ce7a-4a69-9523-5b8ed6b5813b admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:39.527264 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497e01c-ce7a-4a69-9523-5b8ed6b5813b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:39.527958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497e01c-ce7a-4a69-9523-5b8ed6b5813b admin admin] PATH_INFO: `/v3/projects/4e1a1611432f46e1888718a4426b9505/users/0b94c249ff4c4aab8c7238c1ae371ef2/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:39.552205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f497e01c-ce7a-4a69-9523-5b8ed6b5813b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e1a1611432f46e1888718a4426b9505, user_id=0b94c249ff4c4aab8c7238c1ae371ef2, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:39.555736 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f497e01c-ce7a-4a69-9523-5b8ed6b5813b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:39.676015 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 897/1805] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:01:39 2022] PUT /identity/v3/projects/4e1a1611432f46e1888718a4426b9505/users/0b94c249ff4c4aab8c7238c1ae371ef2/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:39.686376 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e73b3c-3825-4a42-88f7-510308f44f75 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:39.686920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e73b3c-3825-4a42-88f7-510308f44f75 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:39.687305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e73b3c-3825-4a42-88f7-510308f44f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:39.722385 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-30e73b3c-3825-4a42-88f7-510308f44f75 None None] MFA Rules not processed for user `0b94c249ff4c4aab8c7238c1ae371ef2`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:39.723635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-30e73b3c-3825-4a42-88f7-510308f44f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:39.792002 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 909/1806] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:39 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:39.796693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f497e01c-ce7a-4a69-9523-5b8ed6b5813b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:39.872181 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f497e01c-ce7a-4a69-9523-5b8ed6b5813b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:39.877373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b855e2-0dd8-413f-aec9-09d5f8c54b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:39.877966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b855e2-0dd8-413f-aec9-09d5f8c54b8d admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:39.878243 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b855e2-0dd8-413f-aec9-09d5f8c54b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:39.878563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b855e2-0dd8-413f-aec9-09d5f8c54b8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:39.879184 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b855e2-0dd8-413f-aec9-09d5f8c54b8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:39.881085 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b855e2-0dd8-413f-aec9-09d5f8c54b8d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:39.985483 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 898/1807] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:01:39 2022] POST /identity/v3/projects => generated 409 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:01:39.989387 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e73b3c-3825-4a42-88f7-510308f44f75 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:39.999755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e73b3c-3825-4a42-88f7-510308f44f75 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:40.061272 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac030b85-a493-4b62-b562-4671d61c7374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:40.061896 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac030b85-a493-4b62-b562-4671d61c7374 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:40.062294 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac030b85-a493-4b62-b562-4671d61c7374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:40.062581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac030b85-a493-4b62-b562-4671d61c7374 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:40.066127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac030b85-a493-4b62-b562-4671d61c7374 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:40.066127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac030b85-a493-4b62-b562-4671d61c7374 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:40.144794 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 910/1808] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:01:39 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:01:40.148936 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b855e2-0dd8-413f-aec9-09d5f8c54b8d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:40.159473 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b855e2-0dd8-413f-aec9-09d5f8c54b8d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:40.164033 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec71672-77c1-4b64-9c72-d6980a2a96d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:40.164787 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec71672-77c1-4b64-9c72-d6980a2a96d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:40.165070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec71672-77c1-4b64-9c72-d6980a2a96d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:40.165345 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec71672-77c1-4b64-9c72-d6980a2a96d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:40.165993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec71672-77c1-4b64-9c72-d6980a2a96d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:40.167645 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec71672-77c1-4b64-9c72-d6980a2a96d8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:40.182034 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 899/1809] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:01:40.184239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac030b85-a493-4b62-b562-4671d61c7374 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:40.205353 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac030b85-a493-4b62-b562-4671d61c7374 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:40.212925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce700db-6554-4704-825e-f2f8ed768cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:40.214750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce700db-6554-4704-825e-f2f8ed768cd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:40.215214 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce700db-6554-4704-825e-f2f8ed768cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:40.216005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce700db-6554-4704-825e-f2f8ed768cd4 admin admin] PATH_INFO: `/v3/projects/5baf5a2c4090421dacbe7d9c08cd215f/users/6cb3b94c97ec4195a535a43d3df478cc/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:40.244877 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:40.248563 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce700db-6554-4704-825e-f2f8ed768cd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5baf5a2c4090421dacbe7d9c08cd215f, user_id=6cb3b94c97ec4195a535a43d3df478cc, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:40.252466 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce700db-6554-4704-825e-f2f8ed768cd4 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:40.329442 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 911/1810] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:01:40 2022] PUT /identity/v3/projects/5baf5a2c4090421dacbe7d9c08cd215f/users/6cb3b94c97ec4195a535a43d3df478cc/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:40.333563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec71672-77c1-4b64-9c72-d6980a2a96d8 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:40.343497 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec71672-77c1-4b64-9c72-d6980a2a96d8 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:40.419034 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e181ba0-3f55-43d2-9edb-391a664d9ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:40.419651 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e181ba0-3f55-43d2-9edb-391a664d9ffb admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:40.419932 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e181ba0-3f55-43d2-9edb-391a664d9ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:40.420201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e181ba0-3f55-43d2-9edb-391a664d9ffb admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:40.420927 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e181ba0-3f55-43d2-9edb-391a664d9ffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:40.422608 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e181ba0-3f55-43d2-9edb-391a664d9ffb admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:40.434736 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 900/1811] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:01:40.438100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce700db-6554-4704-825e-f2f8ed768cd4 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:40.448644 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce700db-6554-4704-825e-f2f8ed768cd4 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:40.454310 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4e696-f98e-434c-ba49-30205fc7e2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:40.455109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4e696-f98e-434c-ba49-30205fc7e2b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:40.455436 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4e696-f98e-434c-ba49-30205fc7e2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:40.455736 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4e696-f98e-434c-ba49-30205fc7e2b1 admin admin] PATH_INFO: `/v3/projects/5baf5a2c4090421dacbe7d9c08cd215f/users/6cb3b94c97ec4195a535a43d3df478cc/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:40.457664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4e696-f98e-434c-ba49-30205fc7e2b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5baf5a2c4090421dacbe7d9c08cd215f, user_id=6cb3b94c97ec4195a535a43d3df478cc, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:40.460460 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4e696-f98e-434c-ba49-30205fc7e2b1 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:40.473160 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 912/1812] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:01:40 2022] PUT /identity/v3/projects/5baf5a2c4090421dacbe7d9c08cd215f/users/6cb3b94c97ec4195a535a43d3df478cc/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:01:40.483511 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3156a-7dba-4d60-a90e-27099f2a79b5 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:40.484181 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3156a-7dba-4d60-a90e-27099f2a79b5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:40.484631 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3156a-7dba-4d60-a90e-27099f2a79b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:40.579146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c4d3156a-7dba-4d60-a90e-27099f2a79b5 None None] MFA Rules not processed for user `6cb3b94c97ec4195a535a43d3df478cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:40.584790 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c4d3156a-7dba-4d60-a90e-27099f2a79b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:40.623195 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:40.698771 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 901/1813] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:40 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:40.710807 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4e696-f98e-434c-ba49-30205fc7e2b1 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:40.723634 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-37f4e696-f98e-434c-ba49-30205fc7e2b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:40.812286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4e696-f98e-434c-ba49-30205fc7e2b1 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:40.822209 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e78f4b-b681-4065-b658-334e6b380355 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:40.823245 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e78f4b-b681-4065-b658-334e6b380355 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:40.823778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e78f4b-b681-4065-b658-334e6b380355 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:40.824153 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e78f4b-b681-4065-b658-334e6b380355 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:40.836872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e78f4b-b681-4065-b658-334e6b380355 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:40.838702 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e78f4b-b681-4065-b658-334e6b380355 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:40.854261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 913/1814] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:40 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:01:41.328261 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3156a-7dba-4d60-a90e-27099f2a79b5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:41.338158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3156a-7dba-4d60-a90e-27099f2a79b5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:41.341940 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d113fa-eee7-4764-a9e5-ca4dbb7aa8f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:41.342637 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d113fa-eee7-4764-a9e5-ca4dbb7aa8f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:41.343001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d113fa-eee7-4764-a9e5-ca4dbb7aa8f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:41.343340 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d113fa-eee7-4764-a9e5-ca4dbb7aa8f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:41.401012 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d113fa-eee7-4764-a9e5-ca4dbb7aa8f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:41.402556 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d113fa-eee7-4764-a9e5-ca4dbb7aa8f2 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:41.411912 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 902/1815] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:41 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:01:50.849803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=35,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:51.407637 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=47 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:56.759160 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f29bfd-7e22-418f-ae8e-2ce2120f678f None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:56.759160 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f29bfd-7e22-418f-ae8e-2ce2120f678f None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:56.759160 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f29bfd-7e22-418f-ae8e-2ce2120f678f None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:56.759160 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 914/1816] 10.222.0.17 () {54 vars in 921 bytes} [Sun Sep 11 00:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:01:56.761335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99f51d-81cd-4ee5-bc63-f101df6cc5f5 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:56.761335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99f51d-81cd-4ee5-bc63-f101df6cc5f5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:56.761335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99f51d-81cd-4ee5-bc63-f101df6cc5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:56.761335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-fe99f51d-81cd-4ee5-bc63-f101df6cc5f5 None None] MFA Rules not processed for user `636bfafb693044d283530063047fc0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:56.761335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fe99f51d-81cd-4ee5-bc63-f101df6cc5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:56.773703 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 903/1817] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Sep 11 00:01:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:01:56.883213 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f29bfd-7e22-418f-ae8e-2ce2120f678f None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:56.884932 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-40f29bfd-7e22-418f-ae8e-2ce2120f678f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:56.905236 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f29bfd-7e22-418f-ae8e-2ce2120f678f None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:56.912403 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350db132-260a-4863-bf14-3588204d6076 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:56.913030 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350db132-260a-4863-bf14-3588204d6076 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:56.913309 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350db132-260a-4863-bf14-3588204d6076 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:56.913582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350db132-260a-4863-bf14-3588204d6076 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:56.935412 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350db132-260a-4863-bf14-3588204d6076 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:56.936817 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350db132-260a-4863-bf14-3588204d6076 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:56.948709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 915/1818] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:01:57.284604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99f51d-81cd-4ee5-bc63-f101df6cc5f5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:57.364052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99f51d-81cd-4ee5-bc63-f101df6cc5f5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:57.368538 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328c264-8c8b-4ee9-a1ad-b83130c4c866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:57.369159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328c264-8c8b-4ee9-a1ad-b83130c4c866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:57.369438 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328c264-8c8b-4ee9-a1ad-b83130c4c866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:57.369708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328c264-8c8b-4ee9-a1ad-b83130c4c866 admin admin] PATH_INFO: `/v3/users/0b94c249ff4c4aab8c7238c1ae371ef2` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:57.370866 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328c264-8c8b-4ee9-a1ad-b83130c4c866 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b94c249ff4c4aab8c7238c1ae371ef2)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:57.372772 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328c264-8c8b-4ee9-a1ad-b83130c4c866 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:57.505765 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c328c264-8c8b-4ee9-a1ad-b83130c4c866 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b94c249ff4c4aab8c7238c1ae371ef2'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:57.518339 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c328c264-8c8b-4ee9-a1ad-b83130c4c866 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b94c249ff4c4aab8c7238c1ae371ef2'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:57.531903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c328c264-8c8b-4ee9-a1ad-b83130c4c866 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b94c249ff4c4aab8c7238c1ae371ef2'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:57.609647 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c328c264-8c8b-4ee9-a1ad-b83130c4c866 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b94c249ff4c4aab8c7238c1ae371ef2'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:57.610264 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c328c264-8c8b-4ee9-a1ad-b83130c4c866 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b94c249ff4c4aab8c7238c1ae371ef2'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:57.617149 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 904/1819] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:01:57 2022] DELETE /identity/v3/users/0b94c249ff4c4aab8c7238c1ae371ef2 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:57.621588 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3c3ae-ea5f-4316-ac01-95b5978537fe None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:57.621960 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3c3ae-ea5f-4316-ac01-95b5978537fe None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:57.622291 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3c3ae-ea5f-4316-ac01-95b5978537fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:57.643881 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-94a3c3ae-ea5f-4316-ac01-95b5978537fe None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:57.645309 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-94a3c3ae-ea5f-4316-ac01-95b5978537fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:57.713052 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 916/1820] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:01:57.724301 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328c264-8c8b-4ee9-a1ad-b83130c4c866 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:57.728168 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c328c264-8c8b-4ee9-a1ad-b83130c4c866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:57.742030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:57.817486 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328c264-8c8b-4ee9-a1ad-b83130c4c866 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:57.821627 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6fcc6-ba4a-4926-90fb-67b18309c357 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:57.828756 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6fcc6-ba4a-4926-90fb-67b18309c357 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:57.828756 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6fcc6-ba4a-4926-90fb-67b18309c357 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:57.828945 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6fcc6-ba4a-4926-90fb-67b18309c357 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:57.861355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de6fcc6-ba4a-4926-90fb-67b18309c357 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:57.861355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de6fcc6-ba4a-4926-90fb-67b18309c357 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:57.869900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 905/1821] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:01:58.354110 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3c3ae-ea5f-4316-ac01-95b5978537fe None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:58.355586 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-94a3c3ae-ea5f-4316-ac01-95b5978537fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:58.368404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3c3ae-ea5f-4316-ac01-95b5978537fe None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:58.373588 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439ecbd-cccf-4188-b63c-37ea7552c9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:58.374220 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439ecbd-cccf-4188-b63c-37ea7552c9d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:58.374845 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439ecbd-cccf-4188-b63c-37ea7552c9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:58.375096 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439ecbd-cccf-4188-b63c-37ea7552c9d7 admin admin] PATH_INFO: `/v3/users/7e8983c2c24941948a58d3c8838e7851` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:58.376298 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439ecbd-cccf-4188-b63c-37ea7552c9d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e8983c2c24941948a58d3c8838e7851)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:58.378528 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439ecbd-cccf-4188-b63c-37ea7552c9d7 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:58.523161 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6fcc6-ba4a-4926-90fb-67b18309c357 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:58.524821 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9de6fcc6-ba4a-4926-90fb-67b18309c357 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:58.545081 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6fcc6-ba4a-4926-90fb-67b18309c357 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:58.552865 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a040a-fc01-4f8b-b180-012dcbfc614c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:58.553841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a040a-fc01-4f8b-b180-012dcbfc614c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:58.554473 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a040a-fc01-4f8b-b180-012dcbfc614c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:58.555039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a040a-fc01-4f8b-b180-012dcbfc614c admin admin] PATH_INFO: `/v3/projects/4e1a1611432f46e1888718a4426b9505` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:58.558869 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a040a-fc01-4f8b-b180-012dcbfc614c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e1a1611432f46e1888718a4426b9505)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:58.568048 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a040a-fc01-4f8b-b180-012dcbfc614c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:58.581233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d439ecbd-cccf-4188-b63c-37ea7552c9d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e8983c2c24941948a58d3c8838e7851'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:58.619140 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d439ecbd-cccf-4188-b63c-37ea7552c9d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e8983c2c24941948a58d3c8838e7851'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:58.635847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d439ecbd-cccf-4188-b63c-37ea7552c9d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e8983c2c24941948a58d3c8838e7851'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:58.638104 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d439ecbd-cccf-4188-b63c-37ea7552c9d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e8983c2c24941948a58d3c8838e7851'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:58.719365 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d439ecbd-cccf-4188-b63c-37ea7552c9d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e8983c2c24941948a58d3c8838e7851'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:58.728271 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 917/1822] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:01:58 2022] DELETE /identity/v3/users/7e8983c2c24941948a58d3c8838e7851 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:58.732937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b6cca-79ce-43b4-8288-fe167944bd91 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:58.733225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b6cca-79ce-43b4-8288-fe167944bd91 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:58.733520 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b6cca-79ce-43b4-8288-fe167944bd91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:58.770216 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-118b6cca-79ce-43b4-8288-fe167944bd91 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:01:58.772706 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-118b6cca-79ce-43b4-8288-fe167944bd91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:58.775249 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ab9a040a-fc01-4f8b-b180-012dcbfc614c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e1a1611432f46e1888718a4426b9505', 'request_id': 'req-ab9a040a-fc01-4f8b-b180-012dcbfc614c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:58.783835 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ab9a040a-fc01-4f8b-b180-012dcbfc614c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e1a1611432f46e1888718a4426b9505', 'request_id': 'req-ab9a040a-fc01-4f8b-b180-012dcbfc614c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:58.792528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ab9a040a-fc01-4f8b-b180-012dcbfc614c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e1a1611432f46e1888718a4426b9505', 'request_id': 'req-ab9a040a-fc01-4f8b-b180-012dcbfc614c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:58.853958 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:58.864413 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 918/1823] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:01:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:01:58.879256 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b6cca-79ce-43b4-8288-fe167944bd91 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:58.879932 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ab9a040a-fc01-4f8b-b180-012dcbfc614c admin admin] The token cache is being invalidate because project 4e1a1611432f46e1888718a4426b9505 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:01:58.880386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ab9a040a-fc01-4f8b-b180-012dcbfc614c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:58.880929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ab9a040a-fc01-4f8b-b180-012dcbfc614c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:58.881346 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-118b6cca-79ce-43b4-8288-fe167944bd91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:58.882426 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 906/1824] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:01:58 2022] DELETE /identity/v3/projects/4e1a1611432f46e1888718a4426b9505 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:01:58.886359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a040a-fc01-4f8b-b180-012dcbfc614c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:58.888030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ab9a040a-fc01-4f8b-b180-012dcbfc614c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:58.901857 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a040a-fc01-4f8b-b180-012dcbfc614c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:58.906856 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca78f801-3063-4149-aa40-e13166d52f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:58.907511 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78f801-3063-4149-aa40-e13166d52f6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:58.907791 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78f801-3063-4149-aa40-e13166d52f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:58.908064 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78f801-3063-4149-aa40-e13166d52f6d admin admin] PATH_INFO: `/v3/users/6cb3b94c97ec4195a535a43d3df478cc` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:58.909232 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca78f801-3063-4149-aa40-e13166d52f6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cb3b94c97ec4195a535a43d3df478cc)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:58.911365 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca78f801-3063-4149-aa40-e13166d52f6d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:58.954766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b6cca-79ce-43b4-8288-fe167944bd91 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:58.959138 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3075d35-0431-4b96-a2dc-624705a70ec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:58.959740 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3075d35-0431-4b96-a2dc-624705a70ec1 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:58.961068 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3075d35-0431-4b96-a2dc-624705a70ec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:58.961068 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3075d35-0431-4b96-a2dc-624705a70ec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:58.962293 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c3075d35-0431-4b96-a2dc-624705a70ec1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:58.974890 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3075d35-0431-4b96-a2dc-624705a70ec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:58.976338 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3075d35-0431-4b96-a2dc-624705a70ec1 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:58.985712 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 919/1825] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:01:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:01:59.207517 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ca78f801-3063-4149-aa40-e13166d52f6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cb3b94c97ec4195a535a43d3df478cc'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:59.216165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ca78f801-3063-4149-aa40-e13166d52f6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cb3b94c97ec4195a535a43d3df478cc'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:59.235119 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ca78f801-3063-4149-aa40-e13166d52f6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cb3b94c97ec4195a535a43d3df478cc'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:59.332125 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ca78f801-3063-4149-aa40-e13166d52f6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cb3b94c97ec4195a535a43d3df478cc'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:59.332841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ca78f801-3063-4149-aa40-e13166d52f6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cb3b94c97ec4195a535a43d3df478cc'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:59.342769 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 907/1826] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:01:58 2022] DELETE /identity/v3/users/6cb3b94c97ec4195a535a43d3df478cc => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:01:59.503992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3075d35-0431-4b96-a2dc-624705a70ec1 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:59.505529 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c3075d35-0431-4b96-a2dc-624705a70ec1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:59.520674 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3075d35-0431-4b96-a2dc-624705a70ec1 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:59.589523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3b29f-c566-4b79-91a1-605bd550ee6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:59.590182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3b29f-c566-4b79-91a1-605bd550ee6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:59.590455 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3b29f-c566-4b79-91a1-605bd550ee6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:59.590759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3b29f-c566-4b79-91a1-605bd550ee6f admin admin] PATH_INFO: `/v3/projects/a9fc207d043e4c19acdf86810c62b55a` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:59.592644 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3b29f-c566-4b79-91a1-605bd550ee6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9fc207d043e4c19acdf86810c62b55a)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:59.600815 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3b29f-c566-4b79-91a1-605bd550ee6f admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:01:59.722746 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d6a3b29f-c566-4b79-91a1-605bd550ee6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9fc207d043e4c19acdf86810c62b55a', 'request_id': 'req-d6a3b29f-c566-4b79-91a1-605bd550ee6f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:59.802260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca78f801-3063-4149-aa40-e13166d52f6d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:59.803733 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ca78f801-3063-4149-aa40-e13166d52f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:59.811363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d6a3b29f-c566-4b79-91a1-605bd550ee6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9fc207d043e4c19acdf86810c62b55a', 'request_id': 'req-d6a3b29f-c566-4b79-91a1-605bd550ee6f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:59.823674 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d6a3b29f-c566-4b79-91a1-605bd550ee6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9fc207d043e4c19acdf86810c62b55a', 'request_id': 'req-d6a3b29f-c566-4b79-91a1-605bd550ee6f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:59.839775 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca78f801-3063-4149-aa40-e13166d52f6d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:59.846099 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d6a3b29f-c566-4b79-91a1-605bd550ee6f admin admin] The token cache is being invalidate because project a9fc207d043e4c19acdf86810c62b55a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:01:59.848902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d6a3b29f-c566-4b79-91a1-605bd550ee6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:59.850021 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d6a3b29f-c566-4b79-91a1-605bd550ee6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:01:59.852945 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:59.858078 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 920/1827] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:01:59 2022] DELETE /identity/v3/projects/a9fc207d043e4c19acdf86810c62b55a => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:01:59.860102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3b29f-c566-4b79-91a1-605bd550ee6f admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:01:59.862955 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-d6a3b29f-c566-4b79-91a1-605bd550ee6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:01:59.915955 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:01:59.957293 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3b29f-c566-4b79-91a1-605bd550ee6f admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:01:59.973718 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888afe6-7c7c-4298-9a46-aab157892ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:01:59.974903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888afe6-7c7c-4298-9a46-aab157892ec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:01:59.978387 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888afe6-7c7c-4298-9a46-aab157892ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:01:59.978387 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888afe6-7c7c-4298-9a46-aab157892ec5 admin admin] PATH_INFO: `/v3/users/51d6b30cd73742b484134f30a96de4b5` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:01:59.978387 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888afe6-7c7c-4298-9a46-aab157892ec5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51d6b30cd73742b484134f30a96de4b5)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:01:59.979474 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888afe6-7c7c-4298-9a46-aab157892ec5 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:00.014245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776b1d2-7a12-4f80-a6b3-36cd79de8fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:00.014245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776b1d2-7a12-4f80-a6b3-36cd79de8fbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:00.014245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776b1d2-7a12-4f80-a6b3-36cd79de8fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:00.014245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776b1d2-7a12-4f80-a6b3-36cd79de8fbd admin admin] PATH_INFO: `/v3/projects/5baf5a2c4090421dacbe7d9c08cd215f` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:00.014922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776b1d2-7a12-4f80-a6b3-36cd79de8fbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5baf5a2c4090421dacbe7d9c08cd215f)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:00.014922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776b1d2-7a12-4f80-a6b3-36cd79de8fbd admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:00.180899 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0776b1d2-7a12-4f80-a6b3-36cd79de8fbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5baf5a2c4090421dacbe7d9c08cd215f', 'request_id': 'req-0776b1d2-7a12-4f80-a6b3-36cd79de8fbd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:00.204025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0776b1d2-7a12-4f80-a6b3-36cd79de8fbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5baf5a2c4090421dacbe7d9c08cd215f', 'request_id': 'req-0776b1d2-7a12-4f80-a6b3-36cd79de8fbd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:00.211282 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0776b1d2-7a12-4f80-a6b3-36cd79de8fbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5baf5a2c4090421dacbe7d9c08cd215f', 'request_id': 'req-0776b1d2-7a12-4f80-a6b3-36cd79de8fbd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:00.337112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0776b1d2-7a12-4f80-a6b3-36cd79de8fbd admin admin] The token cache is being invalidate because project 5baf5a2c4090421dacbe7d9c08cd215f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:00.340185 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0776b1d2-7a12-4f80-a6b3-36cd79de8fbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:00.340185 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0776b1d2-7a12-4f80-a6b3-36cd79de8fbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:00.340185 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 908/1828] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:01:59 2022] DELETE /identity/v3/projects/5baf5a2c4090421dacbe7d9c08cd215f => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:00.344529 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f2ab4-3800-404e-a181-73bca7ad57bf None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:00.345193 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f2ab4-3800-404e-a181-73bca7ad57bf None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:00.345477 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f2ab4-3800-404e-a181-73bca7ad57bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:00.366518 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5f5f2ab4-3800-404e-a181-73bca7ad57bf None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:00.367766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5f5f2ab4-3800-404e-a181-73bca7ad57bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:00.430921 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7888afe6-7c7c-4298-9a46-aab157892ec5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51d6b30cd73742b484134f30a96de4b5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:00.454752 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7888afe6-7c7c-4298-9a46-aab157892ec5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51d6b30cd73742b484134f30a96de4b5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:00.469996 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7888afe6-7c7c-4298-9a46-aab157892ec5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51d6b30cd73742b484134f30a96de4b5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:00.472324 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7888afe6-7c7c-4298-9a46-aab157892ec5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51d6b30cd73742b484134f30a96de4b5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:00.483256 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 909/1829] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:02:00.494054 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca508e-3146-4740-a9af-2f7e7683e868 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:00.494549 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca508e-3146-4740-a9af-2f7e7683e868 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:00.494906 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca508e-3146-4740-a9af-2f7e7683e868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:00.532429 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-acca508e-3146-4740-a9af-2f7e7683e868 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:00.534474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-acca508e-3146-4740-a9af-2f7e7683e868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:00.552000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 910/1830] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:00.553634 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7888afe6-7c7c-4298-9a46-aab157892ec5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51d6b30cd73742b484134f30a96de4b5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:00.554264 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca508e-3146-4740-a9af-2f7e7683e868 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:00.559661 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 921/1831] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:01:59 2022] DELETE /identity/v3/users/51d6b30cd73742b484134f30a96de4b5 => generated 0 bytes in 700 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:02:00.564778 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca508e-3146-4740-a9af-2f7e7683e868 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:00.664323 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fc77c-cb29-4e0e-bb94-dbd9e18221a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:00.664944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fc77c-cb29-4e0e-bb94-dbd9e18221a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:00.665226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fc77c-cb29-4e0e-bb94-dbd9e18221a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:00.665503 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fc77c-cb29-4e0e-bb94-dbd9e18221a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:00.666156 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fc77c-cb29-4e0e-bb94-dbd9e18221a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:02:00.684719 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fc77c-cb29-4e0e-bb94-dbd9e18221a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:00.686975 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fc77c-cb29-4e0e-bb94-dbd9e18221a4 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:00.690701 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:00.706781 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 911/1832] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:02:00.709799 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888afe6-7c7c-4298-9a46-aab157892ec5 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:00.720728 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888afe6-7c7c-4298-9a46-aab157892ec5 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:00.727338 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaddfa0f-02ec-46dd-9267-86ddf29b0bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:00.731743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddfa0f-02ec-46dd-9267-86ddf29b0bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:00.731743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddfa0f-02ec-46dd-9267-86ddf29b0bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:00.731743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddfa0f-02ec-46dd-9267-86ddf29b0bd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:00.731743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaddfa0f-02ec-46dd-9267-86ddf29b0bd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:00.731743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaddfa0f-02ec-46dd-9267-86ddf29b0bd0 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:00.858705 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 922/1833] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:02:00 2022] POST /identity/v3/projects => generated 373 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:00.863488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fc77c-cb29-4e0e-bb94-dbd9e18221a4 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:00.873298 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fc77c-cb29-4e0e-bb94-dbd9e18221a4 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:00.932405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b680727-69c0-4f51-9f9e-b83617ca2366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:00.933056 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b680727-69c0-4f51-9f9e-b83617ca2366 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:00.933226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b680727-69c0-4f51-9f9e-b83617ca2366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:00.933527 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b680727-69c0-4f51-9f9e-b83617ca2366 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:00.934174 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b680727-69c0-4f51-9f9e-b83617ca2366 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:00.936344 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b680727-69c0-4f51-9f9e-b83617ca2366 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:01.054794 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 912/1834] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:02:00 2022] POST /identity/v3/users => generated 287 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:02:01.056012 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaddfa0f-02ec-46dd-9267-86ddf29b0bd0 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:01.066793 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaddfa0f-02ec-46dd-9267-86ddf29b0bd0 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:01.072721 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161ff3b-c8cd-4837-afef-7a58a5801de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:01.073344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161ff3b-c8cd-4837-afef-7a58a5801de4 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:01.073635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161ff3b-c8cd-4837-afef-7a58a5801de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:01.073906 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161ff3b-c8cd-4837-afef-7a58a5801de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:01.074569 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0161ff3b-c8cd-4837-afef-7a58a5801de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:01.076187 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0161ff3b-c8cd-4837-afef-7a58a5801de4 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:01.091820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 923/1835] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:01.093022 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b680727-69c0-4f51-9f9e-b83617ca2366 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:01.103697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b680727-69c0-4f51-9f9e-b83617ca2366 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:01.108429 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708ff31-2a2b-49c9-9692-47ff828dda1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:01.109033 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708ff31-2a2b-49c9-9692-47ff828dda1f admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:01.109312 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708ff31-2a2b-49c9-9692-47ff828dda1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:01.109586 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708ff31-2a2b-49c9-9692-47ff828dda1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:01.110225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708ff31-2a2b-49c9-9692-47ff828dda1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:01.111971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708ff31-2a2b-49c9-9692-47ff828dda1f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:01.122639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 913/1836] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:01.126882 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161ff3b-c8cd-4837-afef-7a58a5801de4 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:01.137213 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161ff3b-c8cd-4837-afef-7a58a5801de4 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:01.142225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f79302-bbf6-4327-8b28-a16ec6513087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:01.142851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f79302-bbf6-4327-8b28-a16ec6513087 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:01.143679 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f79302-bbf6-4327-8b28-a16ec6513087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:01.143951 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f79302-bbf6-4327-8b28-a16ec6513087 admin admin] PATH_INFO: `/v3/projects/15b185408d9440d5a1b60ddb0bd35809/users/a507ed547e534d5e9a0f15d5d7ce9dbd/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:01.157256 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708ff31-2a2b-49c9-9692-47ff828dda1f admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:01.160028 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7708ff31-2a2b-49c9-9692-47ff828dda1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:01.174356 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f79302-bbf6-4327-8b28-a16ec6513087 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15b185408d9440d5a1b60ddb0bd35809, user_id=a507ed547e534d5e9a0f15d5d7ce9dbd, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:01.180674 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f79302-bbf6-4327-8b28-a16ec6513087 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:01.188919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708ff31-2a2b-49c9-9692-47ff828dda1f admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:01.194548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c3da0-a7cc-4567-88f1-a2ae557fdde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:01.195282 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c3da0-a7cc-4567-88f1-a2ae557fdde8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:01.195622 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c3da0-a7cc-4567-88f1-a2ae557fdde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:01.196074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c3da0-a7cc-4567-88f1-a2ae557fdde8 admin admin] PATH_INFO: `/v3/projects/0aa74a0835b44db2856ffb220bc997ea` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:01.197278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c3da0-a7cc-4567-88f1-a2ae557fdde8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0aa74a0835b44db2856ffb220bc997ea)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:01.199297 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c3da0-a7cc-4567-88f1-a2ae557fdde8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:01.262180 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 924/1837] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:01 2022] PUT /identity/v3/projects/15b185408d9440d5a1b60ddb0bd35809/users/a507ed547e534d5e9a0f15d5d7ce9dbd/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:01.267183 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4257431-8f38-433e-89bf-16f35b1240a2 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:01.267522 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4257431-8f38-433e-89bf-16f35b1240a2 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:01.267798 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4257431-8f38-433e-89bf-16f35b1240a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:01.286800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b4257431-8f38-433e-89bf-16f35b1240a2 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:01.288002 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b4257431-8f38-433e-89bf-16f35b1240a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:01.332715 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7c2c3da0-a7cc-4567-88f1-a2ae557fdde8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0aa74a0835b44db2856ffb220bc997ea', 'request_id': 'req-7c2c3da0-a7cc-4567-88f1-a2ae557fdde8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:01.339127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7c2c3da0-a7cc-4567-88f1-a2ae557fdde8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0aa74a0835b44db2856ffb220bc997ea', 'request_id': 'req-7c2c3da0-a7cc-4567-88f1-a2ae557fdde8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:01.344685 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:01.345360 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7c2c3da0-a7cc-4567-88f1-a2ae557fdde8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0aa74a0835b44db2856ffb220bc997ea', 'request_id': 'req-7c2c3da0-a7cc-4567-88f1-a2ae557fdde8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:01.356391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 925/1838] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:02:01.366112 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4257431-8f38-433e-89bf-16f35b1240a2 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:01.367543 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b4257431-8f38-433e-89bf-16f35b1240a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:01.404701 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7c2c3da0-a7cc-4567-88f1-a2ae557fdde8 admin admin] The token cache is being invalidate because project 0aa74a0835b44db2856ffb220bc997ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:01.404914 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7c2c3da0-a7cc-4567-88f1-a2ae557fdde8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:01.405578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7c2c3da0-a7cc-4567-88f1-a2ae557fdde8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:01.411750 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 914/1839] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:02:01 2022] DELETE /identity/v3/projects/0aa74a0835b44db2856ffb220bc997ea => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:02:01.413436 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801e0ad-5afc-45ab-908c-b629e1dabb6c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:01.414014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801e0ad-5afc-45ab-908c-b629e1dabb6c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:01.414402 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801e0ad-5afc-45ab-908c-b629e1dabb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:01.432844 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4257431-8f38-433e-89bf-16f35b1240a2 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:01.436770 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a801e0ad-5afc-45ab-908c-b629e1dabb6c None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:01.438230 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a801e0ad-5afc-45ab-908c-b629e1dabb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:01.492959 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a5f96-cfc2-431c-91b2-57f87c435af1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:01.493863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a5f96-cfc2-431c-91b2-57f87c435af1 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:01.494153 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a5f96-cfc2-431c-91b2-57f87c435af1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:01.494428 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a5f96-cfc2-431c-91b2-57f87c435af1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:01.496500 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-601a5f96-cfc2-431c-91b2-57f87c435af1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:01.517013 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 915/1840] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:02:01.525300 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4b8ed-accf-4dc9-890c-e1625b2abdb6 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:01.525678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4b8ed-accf-4dc9-890c-e1625b2abdb6 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:01.525855 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4b8ed-accf-4dc9-890c-e1625b2abdb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:01.548217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d2e4b8ed-accf-4dc9-890c-e1625b2abdb6 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:01.549475 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d2e4b8ed-accf-4dc9-890c-e1625b2abdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:01.558493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 916/1841] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:02:01.562640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e4b8ed-accf-4dc9-890c-e1625b2abdb6 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:01.574061 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e4b8ed-accf-4dc9-890c-e1625b2abdb6 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:01.575354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a5f96-cfc2-431c-91b2-57f87c435af1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:01.576870 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a5f96-cfc2-431c-91b2-57f87c435af1 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:01.580789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9beecc-8cbd-4f4c-a8c9-c24833e8b4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:01.581401 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9beecc-8cbd-4f4c-a8c9-c24833e8b4ed admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:01.581911 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9beecc-8cbd-4f4c-a8c9-c24833e8b4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:01.582191 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9beecc-8cbd-4f4c-a8c9-c24833e8b4ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:01.582913 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9beecc-8cbd-4f4c-a8c9-c24833e8b4ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:02:01.583527 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9beecc-8cbd-4f4c-a8c9-c24833e8b4ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:01.585334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9beecc-8cbd-4f4c-a8c9-c24833e8b4ed admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:01.588880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 926/1842] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:02:01.604774 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 917/1843] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:02:01.610144 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a5f96-cfc2-431c-91b2-57f87c435af1 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:01.626435 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a5f96-cfc2-431c-91b2-57f87c435af1 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:01.632169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016b542-755a-4ac0-80a0-64dc1b5178cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:01.632866 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016b542-755a-4ac0-80a0-64dc1b5178cf admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:01.633233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016b542-755a-4ac0-80a0-64dc1b5178cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:01.633559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016b542-755a-4ac0-80a0-64dc1b5178cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:01.634326 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8016b542-755a-4ac0-80a0-64dc1b5178cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:01.637805 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8016b542-755a-4ac0-80a0-64dc1b5178cf admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:01.718435 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 927/1844] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:02:01 2022] POST /identity/v3/projects => generated 369 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:01.720294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9beecc-8cbd-4f4c-a8c9-c24833e8b4ed admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:01.759008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9beecc-8cbd-4f4c-a8c9-c24833e8b4ed admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:01.780572 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:01.840635 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279a360-5b6f-4e25-9746-f795d1368d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:01.842915 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279a360-5b6f-4e25-9746-f795d1368d3b admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:01.843285 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279a360-5b6f-4e25-9746-f795d1368d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:01.843832 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279a360-5b6f-4e25-9746-f795d1368d3b admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:01.844578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279a360-5b6f-4e25-9746-f795d1368d3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:01.847816 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279a360-5b6f-4e25-9746-f795d1368d3b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:01.946689 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 918/1845] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:02:01 2022] POST /identity/v3/users => generated 285 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:01.958026 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016b542-755a-4ac0-80a0-64dc1b5178cf admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:01.963294 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016b542-755a-4ac0-80a0-64dc1b5178cf admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:01.967976 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d0dc4-289f-485d-888b-0d19498ab59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:01.968585 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d0dc4-289f-485d-888b-0d19498ab59b admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:01.968889 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d0dc4-289f-485d-888b-0d19498ab59b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:01.969170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d0dc4-289f-485d-888b-0d19498ab59b admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:01.969803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41d0dc4-289f-485d-888b-0d19498ab59b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:01.972105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41d0dc4-289f-485d-888b-0d19498ab59b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:01.993476 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 928/1846] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:01.997500 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279a360-5b6f-4e25-9746-f795d1368d3b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:02.011159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279a360-5b6f-4e25-9746-f795d1368d3b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:02.016065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9a340-a6bf-4fb8-9c8d-12f0688cd576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:02.016676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9a340-a6bf-4fb8-9c8d-12f0688cd576 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:02.016968 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9a340-a6bf-4fb8-9c8d-12f0688cd576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:02.017243 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9a340-a6bf-4fb8-9c8d-12f0688cd576 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:02.017918 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd9a340-a6bf-4fb8-9c8d-12f0688cd576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:02.019677 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd9a340-a6bf-4fb8-9c8d-12f0688cd576 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:02.037670 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 919/1847] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:02.040012 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d0dc4-289f-485d-888b-0d19498ab59b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:02.056259 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d0dc4-289f-485d-888b-0d19498ab59b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:02.063573 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b272e-d09f-42f6-9a07-5c992cf99613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:02.064580 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b272e-d09f-42f6-9a07-5c992cf99613 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:02.065825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b272e-d09f-42f6-9a07-5c992cf99613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:02.066762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b272e-d09f-42f6-9a07-5c992cf99613 admin admin] PATH_INFO: `/v3/projects/8a956c0898ec47c7a42706d27a72d94b/users/1b639ad29baf4862b326e2d234d50452/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:02.109543 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b272e-d09f-42f6-9a07-5c992cf99613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a956c0898ec47c7a42706d27a72d94b, user_id=1b639ad29baf4862b326e2d234d50452, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:02.113169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b272e-d09f-42f6-9a07-5c992cf99613 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:02.264776 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 929/1848] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:02 2022] PUT /identity/v3/projects/8a956c0898ec47c7a42706d27a72d94b/users/1b639ad29baf4862b326e2d234d50452/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:02.277246 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3d33d-a5cd-4101-b5ac-c8aace7607e3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:02.277741 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3d33d-a5cd-4101-b5ac-c8aace7607e3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:02.277925 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3d33d-a5cd-4101-b5ac-c8aace7607e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:02.325667 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-50a3d33d-a5cd-4101-b5ac-c8aace7607e3 None None] MFA Rules not processed for user `1b639ad29baf4862b326e2d234d50452`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:02.326996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-50a3d33d-a5cd-4101-b5ac-c8aace7607e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:02.399712 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 920/1849] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:02.403518 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b272e-d09f-42f6-9a07-5c992cf99613 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:02.410246 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:02.469412 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b272e-d09f-42f6-9a07-5c992cf99613 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:02.475169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb7d80-7ebe-4836-bca5-3f1a9d30f798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:02.475780 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb7d80-7ebe-4836-bca5-3f1a9d30f798 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:02.476058 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb7d80-7ebe-4836-bca5-3f1a9d30f798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:02.476329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb7d80-7ebe-4836-bca5-3f1a9d30f798 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:02.476976 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb7d80-7ebe-4836-bca5-3f1a9d30f798 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:02.478917 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb7d80-7ebe-4836-bca5-3f1a9d30f798 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:02.584356 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 930/1850] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:02:02 2022] POST /identity/v3/projects => generated 365 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:02:02.588355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3d33d-a5cd-4101-b5ac-c8aace7607e3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:02.602313 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3d33d-a5cd-4101-b5ac-c8aace7607e3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:02.664512 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2c9c0-d6ff-4fe1-a655-70db4c7c27e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:02.665121 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2c9c0-d6ff-4fe1-a655-70db4c7c27e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:02.665402 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2c9c0-d6ff-4fe1-a655-70db4c7c27e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:02.665673 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2c9c0-d6ff-4fe1-a655-70db4c7c27e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:02.666390 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c2c9c0-d6ff-4fe1-a655-70db4c7c27e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:02.668395 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c2c9c0-d6ff-4fe1-a655-70db4c7c27e0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:02.673797 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:02.744087 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 921/1851] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:02:02 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:02.748107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb7d80-7ebe-4836-bca5-3f1a9d30f798 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:02.759148 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb7d80-7ebe-4836-bca5-3f1a9d30f798 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:02.764536 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc185d4-0805-42ee-ac53-b91d607d1918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:02.765000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc185d4-0805-42ee-ac53-b91d607d1918 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:02.767182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc185d4-0805-42ee-ac53-b91d607d1918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:02.767182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc185d4-0805-42ee-ac53-b91d607d1918 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:02.767182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc185d4-0805-42ee-ac53-b91d607d1918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:02.768165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc185d4-0805-42ee-ac53-b91d607d1918 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:02.779766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 931/1852] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:02:02.784744 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2c9c0-d6ff-4fe1-a655-70db4c7c27e0 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:02.795286 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2c9c0-d6ff-4fe1-a655-70db4c7c27e0 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:02.800850 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a208822-b16f-4b06-b749-8fb43cb4c413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:02.801636 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a208822-b16f-4b06-b749-8fb43cb4c413 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:02.801992 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a208822-b16f-4b06-b749-8fb43cb4c413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:02.802265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a208822-b16f-4b06-b749-8fb43cb4c413 admin admin] PATH_INFO: `/v3/projects/eecaa089cc2a46559a9b00db5621244a/users/19dbfb73c5c54e7db9b9cc4f74a3cf7d/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:02.820615 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a208822-b16f-4b06-b749-8fb43cb4c413 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eecaa089cc2a46559a9b00db5621244a, user_id=19dbfb73c5c54e7db9b9cc4f74a3cf7d, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:02.824702 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a208822-b16f-4b06-b749-8fb43cb4c413 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:02.901893 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 922/1853] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:02 2022] PUT /identity/v3/projects/eecaa089cc2a46559a9b00db5621244a/users/19dbfb73c5c54e7db9b9cc4f74a3cf7d/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:02:02.908050 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc185d4-0805-42ee-ac53-b91d607d1918 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:02.919649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc185d4-0805-42ee-ac53-b91d607d1918 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:03.003052 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d2a72-5ed5-49f8-8870-53defb522a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:03.004171 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d2a72-5ed5-49f8-8870-53defb522a5d admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:03.005125 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d2a72-5ed5-49f8-8870-53defb522a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:03.005552 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d2a72-5ed5-49f8-8870-53defb522a5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:03.007904 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d2a72-5ed5-49f8-8870-53defb522a5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:03.009851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d2a72-5ed5-49f8-8870-53defb522a5d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:03.027996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a208822-b16f-4b06-b749-8fb43cb4c413 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:03.035155 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 932/1854] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:03.040888 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a208822-b16f-4b06-b749-8fb43cb4c413 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:03.046334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c266848-5a9b-4f2f-94ee-00cdf79bb6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:03.047963 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c266848-5a9b-4f2f-94ee-00cdf79bb6ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:03.048339 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c266848-5a9b-4f2f-94ee-00cdf79bb6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:03.049024 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c266848-5a9b-4f2f-94ee-00cdf79bb6ad admin admin] PATH_INFO: `/v3/projects/eecaa089cc2a46559a9b00db5621244a/users/19dbfb73c5c54e7db9b9cc4f74a3cf7d/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:03.052798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c266848-5a9b-4f2f-94ee-00cdf79bb6ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=eecaa089cc2a46559a9b00db5621244a, user_id=19dbfb73c5c54e7db9b9cc4f74a3cf7d, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:03.055703 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c266848-5a9b-4f2f-94ee-00cdf79bb6ad admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:03.081347 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 923/1855] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:03 2022] PUT /identity/v3/projects/eecaa089cc2a46559a9b00db5621244a/users/19dbfb73c5c54e7db9b9cc4f74a3cf7d/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:03.092219 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94456441-95a8-4a18-b1b1-bd67e7e2b5e6 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:03.092599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94456441-95a8-4a18-b1b1-bd67e7e2b5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:03.093002 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94456441-95a8-4a18-b1b1-bd67e7e2b5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:03.166222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0b6f1-62cd-45b3-9232-74b8ea4b27e8 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:03.166503 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0b6f1-62cd-45b3-9232-74b8ea4b27e8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:03.166996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0b6f1-62cd-45b3-9232-74b8ea4b27e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:03.169299 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-94456441-95a8-4a18-b1b1-bd67e7e2b5e6 None None] MFA Rules not processed for user `19dbfb73c5c54e7db9b9cc4f74a3cf7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:03.170579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-94456441-95a8-4a18-b1b1-bd67e7e2b5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:03.204439 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-acf0b6f1-62cd-45b3-9232-74b8ea4b27e8 None None] MFA Rules not processed for user `a507ed547e534d5e9a0f15d5d7ce9dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:03.206692 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-acf0b6f1-62cd-45b3-9232-74b8ea4b27e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:03.274972 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:03.280392 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 933/1856] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:03 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:03.309592 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94456441-95a8-4a18-b1b1-bd67e7e2b5e6 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:03.312350 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-94456441-95a8-4a18-b1b1-bd67e7e2b5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:03.397996 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94456441-95a8-4a18-b1b1-bd67e7e2b5e6 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:03.401644 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844d58c-7569-4559-9430-7917991f4642 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:03.402264 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844d58c-7569-4559-9430-7917991f4642 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:03.402541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844d58c-7569-4559-9430-7917991f4642 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:03.402981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844d58c-7569-4559-9430-7917991f4642 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:03.415336 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 924/1857] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:02:03.419657 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e844d58c-7569-4559-9430-7917991f4642 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:03.421108 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e844d58c-7569-4559-9430-7917991f4642 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:03.423541 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf0b6f1-62cd-45b3-9232-74b8ea4b27e8 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:03.426769 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-acf0b6f1-62cd-45b3-9232-74b8ea4b27e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:03.451153 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 934/1858] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:03 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:02:03.528400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf0b6f1-62cd-45b3-9232-74b8ea4b27e8 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:03.533035 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b5964-da6c-46f1-97dc-84e881f606d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:03.534251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b5964-da6c-46f1-97dc-84e881f606d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:03.534633 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b5964-da6c-46f1-97dc-84e881f606d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:03.534921 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b5964-da6c-46f1-97dc-84e881f606d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:03.535643 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b5964-da6c-46f1-97dc-84e881f606d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:03.537635 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b5964-da6c-46f1-97dc-84e881f606d0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:03.542949 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:03.640140 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 925/1859] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:02:03 2022] POST /identity/v3/projects => generated 373 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:03.644454 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844d58c-7569-4559-9430-7917991f4642 service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:03.656982 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844d58c-7569-4559-9430-7917991f4642 service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:03.685861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b5964-da6c-46f1-97dc-84e881f606d0 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:03.687619 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-005b5964-da6c-46f1-97dc-84e881f606d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:03.726254 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3bbf28-6f67-477e-ab03-dbf790b1f5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:03.727032 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3bbf28-6f67-477e-ab03-dbf790b1f5fb admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:03.727306 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3bbf28-6f67-477e-ab03-dbf790b1f5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:03.727572 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3bbf28-6f67-477e-ab03-dbf790b1f5fb admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:03.735616 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3bbf28-6f67-477e-ab03-dbf790b1f5fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:03.737702 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3bbf28-6f67-477e-ab03-dbf790b1f5fb admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:03.751864 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b5964-da6c-46f1-97dc-84e881f606d0 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:03.756597 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8572f82b-9eaa-4121-91cb-3c977f7b031b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:03.757208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572f82b-9eaa-4121-91cb-3c977f7b031b service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:03.757483 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572f82b-9eaa-4121-91cb-3c977f7b031b service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:03.757749 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572f82b-9eaa-4121-91cb-3c977f7b031b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:03.828445 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 935/1860] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:02:03 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:02:03.831892 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8572f82b-9eaa-4121-91cb-3c977f7b031b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:03.833011 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3bbf28-6f67-477e-ab03-dbf790b1f5fb admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:03.833918 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8572f82b-9eaa-4121-91cb-3c977f7b031b service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:03.843279 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 926/1861] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:02:03.844856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3bbf28-6f67-477e-ab03-dbf790b1f5fb admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:03.853485 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feaac83-911b-4ba7-b97a-7ba0be9eafa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:03.854095 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaac83-911b-4ba7-b97a-7ba0be9eafa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:03.854365 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaac83-911b-4ba7-b97a-7ba0be9eafa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:03.854908 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaac83-911b-4ba7-b97a-7ba0be9eafa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:03.858800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feaac83-911b-4ba7-b97a-7ba0be9eafa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:03.858800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feaac83-911b-4ba7-b97a-7ba0be9eafa1 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:03.875150 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 936/1862] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:03.879205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8572f82b-9eaa-4121-91cb-3c977f7b031b service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:03.891322 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8572f82b-9eaa-4121-91cb-3c977f7b031b service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:03.897262 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e2a75-1469-4eeb-a242-ef0f64a64c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:03.898165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e2a75-1469-4eeb-a242-ef0f64a64c56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:03.898446 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e2a75-1469-4eeb-a242-ef0f64a64c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:03.898744 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e2a75-1469-4eeb-a242-ef0f64a64c56 admin admin] PATH_INFO: `/v3/projects/1ead4d3719db4002b3be221143d12994/users/e24e5ae5119848cb9b430ba243037c74/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:03.916079 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139e2a75-1469-4eeb-a242-ef0f64a64c56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ead4d3719db4002b3be221143d12994, user_id=e24e5ae5119848cb9b430ba243037c74, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:03.920936 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139e2a75-1469-4eeb-a242-ef0f64a64c56 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:03.990243 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 927/1863] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:03 2022] PUT /identity/v3/projects/1ead4d3719db4002b3be221143d12994/users/e24e5ae5119848cb9b430ba243037c74/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:03.994734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feaac83-911b-4ba7-b97a-7ba0be9eafa1 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:04.006610 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feaac83-911b-4ba7-b97a-7ba0be9eafa1 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:04.065712 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337ce09-370d-4330-9f79-8bde18216147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:04.066336 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337ce09-370d-4330-9f79-8bde18216147 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:04.066613 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337ce09-370d-4330-9f79-8bde18216147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:04.069604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337ce09-370d-4330-9f79-8bde18216147 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:04.069604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337ce09-370d-4330-9f79-8bde18216147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:04.069604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337ce09-370d-4330-9f79-8bde18216147 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:04.079134 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 937/1864] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:04.083119 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e2a75-1469-4eeb-a242-ef0f64a64c56 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:04.093294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e2a75-1469-4eeb-a242-ef0f64a64c56 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:04.098330 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59493a07-06fd-4110-b4fa-8cf9b75006ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:04.099009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59493a07-06fd-4110-b4fa-8cf9b75006ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:04.099289 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59493a07-06fd-4110-b4fa-8cf9b75006ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:04.099561 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59493a07-06fd-4110-b4fa-8cf9b75006ff admin admin] PATH_INFO: `/v3/projects/1ead4d3719db4002b3be221143d12994/users/e24e5ae5119848cb9b430ba243037c74/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:04.101443 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59493a07-06fd-4110-b4fa-8cf9b75006ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ead4d3719db4002b3be221143d12994, user_id=e24e5ae5119848cb9b430ba243037c74, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:04.104146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59493a07-06fd-4110-b4fa-8cf9b75006ff admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:04.116122 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 928/1865] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:04 2022] PUT /identity/v3/projects/1ead4d3719db4002b3be221143d12994/users/e24e5ae5119848cb9b430ba243037c74/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:04.934558 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337ce09-370d-4330-9f79-8bde18216147 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:04.936702 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2337ce09-370d-4330-9f79-8bde18216147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:04.957185 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337ce09-370d-4330-9f79-8bde18216147 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:04.973177 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:05.027118 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f271cfcf-8242-4506-b466-627b10c7bf6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:05.027758 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271cfcf-8242-4506-b466-627b10c7bf6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:05.028069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271cfcf-8242-4506-b466-627b10c7bf6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:05.028654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271cfcf-8242-4506-b466-627b10c7bf6f admin admin] PATH_INFO: `/v3/users/6abd75ccff2241c3bf344c4ce9b34e55` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:05.029831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f271cfcf-8242-4506-b466-627b10c7bf6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6abd75ccff2241c3bf344c4ce9b34e55)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:05.031893 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f271cfcf-8242-4506-b466-627b10c7bf6f admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:05.290588 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f271cfcf-8242-4506-b466-627b10c7bf6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6abd75ccff2241c3bf344c4ce9b34e55'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:05.303593 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f271cfcf-8242-4506-b466-627b10c7bf6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6abd75ccff2241c3bf344c4ce9b34e55'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:05.317116 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f271cfcf-8242-4506-b466-627b10c7bf6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6abd75ccff2241c3bf344c4ce9b34e55'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:05.318443 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f271cfcf-8242-4506-b466-627b10c7bf6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6abd75ccff2241c3bf344c4ce9b34e55'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:05.376591 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f271cfcf-8242-4506-b466-627b10c7bf6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6abd75ccff2241c3bf344c4ce9b34e55'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:05.385233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 938/1866] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:02:04 2022] DELETE /identity/v3/users/6abd75ccff2241c3bf344c4ce9b34e55 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:05.389790 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4b267-a12e-4a89-84d0-a9c01eedb780 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:05.390781 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4b267-a12e-4a89-84d0-a9c01eedb780 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:05.391195 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4b267-a12e-4a89-84d0-a9c01eedb780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:05.415546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b8e4b267-a12e-4a89-84d0-a9c01eedb780 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:05.416878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b8e4b267-a12e-4a89-84d0-a9c01eedb780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:05.503737 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 929/1867] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:02:05.514275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f271cfcf-8242-4506-b466-627b10c7bf6f admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:05.515745 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f271cfcf-8242-4506-b466-627b10c7bf6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:05.588983 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f271cfcf-8242-4506-b466-627b10c7bf6f admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:05.593233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7064b0-fc5c-4949-9f46-092b54b7d0f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:05.593860 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7064b0-fc5c-4949-9f46-092b54b7d0f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:05.594138 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7064b0-fc5c-4949-9f46-092b54b7d0f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:05.594408 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7064b0-fc5c-4949-9f46-092b54b7d0f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:05.605570 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7064b0-fc5c-4949-9f46-092b54b7d0f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:05.607120 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7064b0-fc5c-4949-9f46-092b54b7d0f9 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:05.619000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 939/1868] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:02:05.711299 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52803b-5859-4f2c-890e-cf3b5ec8e02f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:05.711299 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52803b-5859-4f2c-890e-cf3b5ec8e02f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:05.711299 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52803b-5859-4f2c-890e-cf3b5ec8e02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:05.719802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:05.754996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ef52803b-5859-4f2c-890e-cf3b5ec8e02f None None] MFA Rules not processed for user `e24e5ae5119848cb9b430ba243037c74`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:05.756339 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ef52803b-5859-4f2c-890e-cf3b5ec8e02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:05.860869 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 930/1869] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:05.869937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7064b0-fc5c-4949-9f46-092b54b7d0f9 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:05.873262 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3f7064b0-fc5c-4949-9f46-092b54b7d0f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:05.932790 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:05.948052 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7064b0-fc5c-4949-9f46-092b54b7d0f9 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:05.951857 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca2d758-f7fb-4d0e-9fa1-d8d7d88db75c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:05.952473 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca2d758-f7fb-4d0e-9fa1-d8d7d88db75c service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:05.952749 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca2d758-f7fb-4d0e-9fa1-d8d7d88db75c service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:05.953017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca2d758-f7fb-4d0e-9fa1-d8d7d88db75c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:05.955292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cca2d758-f7fb-4d0e-9fa1-d8d7d88db75c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:06.041203 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca2d758-f7fb-4d0e-9fa1-d8d7d88db75c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:06.043069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca2d758-f7fb-4d0e-9fa1-d8d7d88db75c service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:06.054770 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 940/1870] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:02:06.231870 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52803b-5859-4f2c-890e-cf3b5ec8e02f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:06.234077 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ef52803b-5859-4f2c-890e-cf3b5ec8e02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:06.255688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52803b-5859-4f2c-890e-cf3b5ec8e02f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:06.260155 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abe3d4-1f6e-4fbd-9678-ebb6e7d8da72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:06.260794 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abe3d4-1f6e-4fbd-9678-ebb6e7d8da72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:06.261080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abe3d4-1f6e-4fbd-9678-ebb6e7d8da72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:06.262071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abe3d4-1f6e-4fbd-9678-ebb6e7d8da72 admin admin] PATH_INFO: `/v3/projects/4cd2dbd4a1b442eb86a3d663a9bfd1c1` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:06.263420 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10abe3d4-1f6e-4fbd-9678-ebb6e7d8da72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cd2dbd4a1b442eb86a3d663a9bfd1c1)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:06.265362 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10abe3d4-1f6e-4fbd-9678-ebb6e7d8da72 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:06.400674 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-10abe3d4-1f6e-4fbd-9678-ebb6e7d8da72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cd2dbd4a1b442eb86a3d663a9bfd1c1', 'request_id': 'req-10abe3d4-1f6e-4fbd-9678-ebb6e7d8da72'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:06.407968 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-10abe3d4-1f6e-4fbd-9678-ebb6e7d8da72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cd2dbd4a1b442eb86a3d663a9bfd1c1', 'request_id': 'req-10abe3d4-1f6e-4fbd-9678-ebb6e7d8da72'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:06.414138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-10abe3d4-1f6e-4fbd-9678-ebb6e7d8da72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cd2dbd4a1b442eb86a3d663a9bfd1c1', 'request_id': 'req-10abe3d4-1f6e-4fbd-9678-ebb6e7d8da72'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:06.467104 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-10abe3d4-1f6e-4fbd-9678-ebb6e7d8da72 admin admin] The token cache is being invalidate because project 4cd2dbd4a1b442eb86a3d663a9bfd1c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:06.467419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-10abe3d4-1f6e-4fbd-9678-ebb6e7d8da72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:06.468058 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-10abe3d4-1f6e-4fbd-9678-ebb6e7d8da72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:06.469602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 931/1871] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:02:06 2022] DELETE /identity/v3/projects/4cd2dbd4a1b442eb86a3d663a9bfd1c1 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:06.473795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca2d758-f7fb-4d0e-9fa1-d8d7d88db75c service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:06.484510 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca2d758-f7fb-4d0e-9fa1-d8d7d88db75c service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:06.547652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37de8b7-4f54-45de-a7e6-e906640136d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:06.548290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37de8b7-4f54-45de-a7e6-e906640136d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:06.548566 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37de8b7-4f54-45de-a7e6-e906640136d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:06.548834 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37de8b7-4f54-45de-a7e6-e906640136d7 admin admin] PATH_INFO: `/v3/users/62d3f21c326e4ba39b165d74e8feb5d3` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:06.551021 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37de8b7-4f54-45de-a7e6-e906640136d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62d3f21c326e4ba39b165d74e8feb5d3)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:06.554755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37de8b7-4f54-45de-a7e6-e906640136d7 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:06.744779 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c37de8b7-4f54-45de-a7e6-e906640136d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62d3f21c326e4ba39b165d74e8feb5d3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:06.756418 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c37de8b7-4f54-45de-a7e6-e906640136d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62d3f21c326e4ba39b165d74e8feb5d3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:06.763094 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c37de8b7-4f54-45de-a7e6-e906640136d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62d3f21c326e4ba39b165d74e8feb5d3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:06.763733 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c37de8b7-4f54-45de-a7e6-e906640136d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62d3f21c326e4ba39b165d74e8feb5d3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:06.856129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c37de8b7-4f54-45de-a7e6-e906640136d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62d3f21c326e4ba39b165d74e8feb5d3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:06.863331 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 941/1872] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:02:06 2022] DELETE /identity/v3/users/62d3f21c326e4ba39b165d74e8feb5d3 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:07.371115 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abe3d4-1f6e-4fbd-9678-ebb6e7d8da72 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:07.373110 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-10abe3d4-1f6e-4fbd-9678-ebb6e7d8da72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:07.387399 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abe3d4-1f6e-4fbd-9678-ebb6e7d8da72 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:07.455153 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78197b-75bb-472a-95b1-3f4646616af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:07.457218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78197b-75bb-472a-95b1-3f4646616af6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:07.457659 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78197b-75bb-472a-95b1-3f4646616af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:07.458396 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78197b-75bb-472a-95b1-3f4646616af6 admin admin] PATH_INFO: `/v3/projects/8375da4aa42444cabc5b0511a06632ee` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:07.460506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78197b-75bb-472a-95b1-3f4646616af6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8375da4aa42444cabc5b0511a06632ee)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:07.463390 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78197b-75bb-472a-95b1-3f4646616af6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:07.488153 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:07.586226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fe78197b-75bb-472a-95b1-3f4646616af6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8375da4aa42444cabc5b0511a06632ee', 'request_id': 'req-fe78197b-75bb-472a-95b1-3f4646616af6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:07.593525 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fe78197b-75bb-472a-95b1-3f4646616af6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8375da4aa42444cabc5b0511a06632ee', 'request_id': 'req-fe78197b-75bb-472a-95b1-3f4646616af6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:07.600448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fe78197b-75bb-472a-95b1-3f4646616af6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8375da4aa42444cabc5b0511a06632ee', 'request_id': 'req-fe78197b-75bb-472a-95b1-3f4646616af6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:07.685752 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fe78197b-75bb-472a-95b1-3f4646616af6 admin admin] The token cache is being invalidate because project 8375da4aa42444cabc5b0511a06632ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:07.686045 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fe78197b-75bb-472a-95b1-3f4646616af6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:07.686614 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fe78197b-75bb-472a-95b1-3f4646616af6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:07.688135 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 932/1873] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:02:07 2022] DELETE /identity/v3/projects/8375da4aa42444cabc5b0511a06632ee => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:07.694118 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3992bdab-45be-4cd0-b262-7448eed5fcf4 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:07.694640 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3992bdab-45be-4cd0-b262-7448eed5fcf4 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:07.694802 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3992bdab-45be-4cd0-b262-7448eed5fcf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:07.714692 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-3992bdab-45be-4cd0-b262-7448eed5fcf4 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:07.715875 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3992bdab-45be-4cd0-b262-7448eed5fcf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:07.724652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:07.779993 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 942/1874] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:07.788220 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8fab1-1f51-4d7d-b64a-bf575d2fe536 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:07.788504 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8fab1-1f51-4d7d-b64a-bf575d2fe536 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:07.788782 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8fab1-1f51-4d7d-b64a-bf575d2fe536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:07.809333 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-bdf8fab1-1f51-4d7d-b64a-bf575d2fe536 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:07.810610 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-bdf8fab1-1f51-4d7d-b64a-bf575d2fe536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:07.818971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 933/1875] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:07.823401 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3992bdab-45be-4cd0-b262-7448eed5fcf4 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:07.833972 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3992bdab-45be-4cd0-b262-7448eed5fcf4 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:07.840084 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d3e85-0dcb-44b5-bfc3-a2d66ca41125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:07.840084 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d3e85-0dcb-44b5-bfc3-a2d66ca41125 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:07.841617 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d3e85-0dcb-44b5-bfc3-a2d66ca41125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:07.841617 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d3e85-0dcb-44b5-bfc3-a2d66ca41125 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:07.841617 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d3e85-0dcb-44b5-bfc3-a2d66ca41125 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:02:07.841617 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d3e85-0dcb-44b5-bfc3-a2d66ca41125 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:07.842969 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d3e85-0dcb-44b5-bfc3-a2d66ca41125 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:07.858517 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 943/1876] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:02:07.860396 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8fab1-1f51-4d7d-b64a-bf575d2fe536 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:07.869750 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8fab1-1f51-4d7d-b64a-bf575d2fe536 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:07.874348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084d04d-690d-4b66-9e2e-54d1877ca308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:07.875017 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084d04d-690d-4b66-9e2e-54d1877ca308 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:07.875287 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084d04d-690d-4b66-9e2e-54d1877ca308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:07.875570 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084d04d-690d-4b66-9e2e-54d1877ca308 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:07.876221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084d04d-690d-4b66-9e2e-54d1877ca308 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:07.878897 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084d04d-690d-4b66-9e2e-54d1877ca308 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:07.970852 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d3e85-0dcb-44b5-bfc3-a2d66ca41125 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:07.972789 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-578d3e85-0dcb-44b5-bfc3-a2d66ca41125 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:07.992957 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 934/1877] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:02:07 2022] POST /identity/v3/projects => generated 363 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:07.994802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084d04d-690d-4b66-9e2e-54d1877ca308 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:08.047580 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d3e85-0dcb-44b5-bfc3-a2d66ca41125 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:08.085964 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084d04d-690d-4b66-9e2e-54d1877ca308 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:08.091575 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee692cc-ac1c-44ae-990e-225642fd5507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:08.092311 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee692cc-ac1c-44ae-990e-225642fd5507 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:08.092693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee692cc-ac1c-44ae-990e-225642fd5507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:08.093068 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee692cc-ac1c-44ae-990e-225642fd5507 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:08.093775 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee692cc-ac1c-44ae-990e-225642fd5507 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:08.096590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee692cc-ac1c-44ae-990e-225642fd5507 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:08.118083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0284722-89a3-4589-81d4-149f1aeb0923 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:08.118718 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0284722-89a3-4589-81d4-149f1aeb0923 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:08.119115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0284722-89a3-4589-81d4-149f1aeb0923 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:08.119387 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0284722-89a3-4589-81d4-149f1aeb0923 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:08.121219 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c0284722-89a3-4589-81d4-149f1aeb0923 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:08.188498 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 935/1878] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:02:07 2022] POST /identity/v3/users => generated 282 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:02:08.194467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee692cc-ac1c-44ae-990e-225642fd5507 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:08.211595 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0284722-89a3-4589-81d4-149f1aeb0923 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:08.211595 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0284722-89a3-4589-81d4-149f1aeb0923 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:08.211804 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee692cc-ac1c-44ae-990e-225642fd5507 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:08.213898 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5bec5-4f7f-46fc-baae-b417a366f75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:08.214685 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5bec5-4f7f-46fc-baae-b417a366f75c admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:08.215002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5bec5-4f7f-46fc-baae-b417a366f75c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:08.215386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5bec5-4f7f-46fc-baae-b417a366f75c admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:08.216313 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5bec5-4f7f-46fc-baae-b417a366f75c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:08.218039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5bec5-4f7f-46fc-baae-b417a366f75c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:08.234533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 944/1880] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:02:08.234977 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 936/1880] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:08.239614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0284722-89a3-4589-81d4-149f1aeb0923 service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:08.258005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0284722-89a3-4589-81d4-149f1aeb0923 service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:08.266986 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410821a-5acd-42bc-9c20-f38ee90ef61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:08.268077 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410821a-5acd-42bc-9c20-f38ee90ef61f admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:08.268544 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410821a-5acd-42bc-9c20-f38ee90ef61f admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:08.269033 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410821a-5acd-42bc-9c20-f38ee90ef61f admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:08.270189 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2410821a-5acd-42bc-9c20-f38ee90ef61f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:08.272484 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2410821a-5acd-42bc-9c20-f38ee90ef61f admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:08.274139 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:08.284950 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 945/1881] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:08.289352 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5bec5-4f7f-46fc-baae-b417a366f75c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:08.300611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5bec5-4f7f-46fc-baae-b417a366f75c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:08.306337 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01edb158-5f66-4e08-80fe-d63a39986f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:08.307117 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edb158-5f66-4e08-80fe-d63a39986f80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:08.310802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edb158-5f66-4e08-80fe-d63a39986f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:08.310802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edb158-5f66-4e08-80fe-d63a39986f80 admin admin] PATH_INFO: `/v3/projects/5f2005ed586648d1b13872c45324b5f0/users/82e761c620704517a292482256589d08/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:08.325584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01edb158-5f66-4e08-80fe-d63a39986f80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f2005ed586648d1b13872c45324b5f0, user_id=82e761c620704517a292482256589d08, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:08.328319 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01edb158-5f66-4e08-80fe-d63a39986f80 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:08.398614 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 937/1882] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:08 2022] PUT /identity/v3/projects/5f2005ed586648d1b13872c45324b5f0/users/82e761c620704517a292482256589d08/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:02:08.409388 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84d95f-59d2-4c77-8c9b-3fc3ff5b9d87 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:08.409787 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84d95f-59d2-4c77-8c9b-3fc3ff5b9d87 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:08.410062 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84d95f-59d2-4c77-8c9b-3fc3ff5b9d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:08.465839 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-0a84d95f-59d2-4c77-8c9b-3fc3ff5b9d87 None None] MFA Rules not processed for user `82e761c620704517a292482256589d08`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:08.465839 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0a84d95f-59d2-4c77-8c9b-3fc3ff5b9d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:08.543727 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 946/1883] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:08.548943 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01edb158-5f66-4e08-80fe-d63a39986f80 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:08.597864 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:08.640172 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01edb158-5f66-4e08-80fe-d63a39986f80 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:08.645738 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a2a6c-0fd0-4827-9be2-de70b0a9cbf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:08.646524 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a2a6c-0fd0-4827-9be2-de70b0a9cbf1 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:08.647057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a2a6c-0fd0-4827-9be2-de70b0a9cbf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:08.647425 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a2a6c-0fd0-4827-9be2-de70b0a9cbf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:08.649713 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a2a6c-0fd0-4827-9be2-de70b0a9cbf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:08.650385 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a2a6c-0fd0-4827-9be2-de70b0a9cbf1 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:08.781375 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 938/1884] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:02:08 2022] POST /identity/v3/projects => generated 365 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:08.786886 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84d95f-59d2-4c77-8c9b-3fc3ff5b9d87 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:08.805896 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84d95f-59d2-4c77-8c9b-3fc3ff5b9d87 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:08.920818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d150b18-0d19-4e6a-9fd0-59a10f13598f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:08.921424 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d150b18-0d19-4e6a-9fd0-59a10f13598f admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:08.921699 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d150b18-0d19-4e6a-9fd0-59a10f13598f admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:08.921965 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d150b18-0d19-4e6a-9fd0-59a10f13598f admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:08.922609 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d150b18-0d19-4e6a-9fd0-59a10f13598f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:08.925097 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d150b18-0d19-4e6a-9fd0-59a10f13598f admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:09.007413 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 947/1885] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:02:08 2022] POST /identity/v3/users => generated 289 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:09.012653 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a2a6c-0fd0-4827-9be2-de70b0a9cbf1 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:09.026387 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a2a6c-0fd0-4827-9be2-de70b0a9cbf1 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:09.035906 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fcaae4-bc87-4e85-a775-e57e3799a480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:09.036839 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fcaae4-bc87-4e85-a775-e57e3799a480 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:09.037190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fcaae4-bc87-4e85-a775-e57e3799a480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:09.037568 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fcaae4-bc87-4e85-a775-e57e3799a480 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:09.038294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fcaae4-bc87-4e85-a775-e57e3799a480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:09.040949 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fcaae4-bc87-4e85-a775-e57e3799a480 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:09.054872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 939/1886] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:09.059699 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d150b18-0d19-4e6a-9fd0-59a10f13598f admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:09.071549 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d150b18-0d19-4e6a-9fd0-59a10f13598f admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:09.076655 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bbf035-1b47-4908-a8c1-2ae5ab99ea53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:09.077320 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbf035-1b47-4908-a8c1-2ae5ab99ea53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:09.077600 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbf035-1b47-4908-a8c1-2ae5ab99ea53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:09.077876 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbf035-1b47-4908-a8c1-2ae5ab99ea53 admin admin] PATH_INFO: `/v3/projects/7f964f2ae98e4e5aaf778cdcaddae72a/users/7d02c6d1d7f440f18a692e53de82947a/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:09.094538 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bbf035-1b47-4908-a8c1-2ae5ab99ea53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f964f2ae98e4e5aaf778cdcaddae72a, user_id=7d02c6d1d7f440f18a692e53de82947a, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:09.097547 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bbf035-1b47-4908-a8c1-2ae5ab99ea53 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:09.178653 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 948/1887] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:09 2022] PUT /identity/v3/projects/7f964f2ae98e4e5aaf778cdcaddae72a/users/7d02c6d1d7f440f18a692e53de82947a/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:09.182975 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fcaae4-bc87-4e85-a775-e57e3799a480 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:09.195349 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fcaae4-bc87-4e85-a775-e57e3799a480 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:09.265689 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c100ce-16e8-40ca-a9dc-21e1c3875a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:09.266216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c100ce-16e8-40ca-a9dc-21e1c3875a15 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:09.266501 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c100ce-16e8-40ca-a9dc-21e1c3875a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:09.267311 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c100ce-16e8-40ca-a9dc-21e1c3875a15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:09.268056 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c100ce-16e8-40ca-a9dc-21e1c3875a15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:09.270692 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c100ce-16e8-40ca-a9dc-21e1c3875a15 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:09.288278 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bbf035-1b47-4908-a8c1-2ae5ab99ea53 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:09.289328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 940/1888] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:09.305139 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bbf035-1b47-4908-a8c1-2ae5ab99ea53 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:09.310199 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf631d3-656e-49f2-9e2d-d1624887cc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:09.310959 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf631d3-656e-49f2-9e2d-d1624887cc3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:09.311237 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf631d3-656e-49f2-9e2d-d1624887cc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:09.311511 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf631d3-656e-49f2-9e2d-d1624887cc3d admin admin] PATH_INFO: `/v3/projects/7f964f2ae98e4e5aaf778cdcaddae72a/users/7d02c6d1d7f440f18a692e53de82947a/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:09.313504 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf631d3-656e-49f2-9e2d-d1624887cc3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f964f2ae98e4e5aaf778cdcaddae72a, user_id=7d02c6d1d7f440f18a692e53de82947a, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:09.317015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf631d3-656e-49f2-9e2d-d1624887cc3d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:09.339404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 949/1889] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:09 2022] PUT /identity/v3/projects/7f964f2ae98e4e5aaf778cdcaddae72a/users/7d02c6d1d7f440f18a692e53de82947a/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:09.341267 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41fb74-1062-402f-b58f-8c001c408b5e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:09.341546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41fb74-1062-402f-b58f-8c001c408b5e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:09.341816 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41fb74-1062-402f-b58f-8c001c408b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:09.376693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-dc41fb74-1062-402f-b58f-8c001c408b5e None None] MFA Rules not processed for user `7d02c6d1d7f440f18a692e53de82947a`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:09.378026 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-dc41fb74-1062-402f-b58f-8c001c408b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:09.487330 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 941/1890] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:09 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:09.499056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf631d3-656e-49f2-9e2d-d1624887cc3d admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:09.554326 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:09.568557 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf631d3-656e-49f2-9e2d-d1624887cc3d admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:09.572236 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374a0ee-7b4c-43d4-b50b-42f80361723f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:09.572844 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374a0ee-7b4c-43d4-b50b-42f80361723f service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:09.573122 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374a0ee-7b4c-43d4-b50b-42f80361723f service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:09.573393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374a0ee-7b4c-43d4-b50b-42f80361723f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:09.583582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8374a0ee-7b4c-43d4-b50b-42f80361723f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:09.586970 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8374a0ee-7b4c-43d4-b50b-42f80361723f service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:09.595201 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 950/1891] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:09 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:02:09.626385 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41fb74-1062-402f-b58f-8c001c408b5e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:09.628692 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-dc41fb74-1062-402f-b58f-8c001c408b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:09.655329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:09.701404 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41fb74-1062-402f-b58f-8c001c408b5e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:09.707977 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212c7a5-c332-4323-b6da-c0fc58032e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:09.711725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212c7a5-c332-4323-b6da-c0fc58032e30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:09.711725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212c7a5-c332-4323-b6da-c0fc58032e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:09.711725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212c7a5-c332-4323-b6da-c0fc58032e30 admin admin] PATH_INFO: `/v3/users/1b639ad29baf4862b326e2d234d50452` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:09.712901 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c212c7a5-c332-4323-b6da-c0fc58032e30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b639ad29baf4862b326e2d234d50452)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:09.714724 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c212c7a5-c332-4323-b6da-c0fc58032e30 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:09.826024 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374a0ee-7b4c-43d4-b50b-42f80361723f service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:09.827579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8374a0ee-7b4c-43d4-b50b-42f80361723f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:09.839990 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374a0ee-7b4c-43d4-b50b-42f80361723f service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:09.844148 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b21a63-b420-43bd-927e-42a699bd472c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:09.844924 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b21a63-b420-43bd-927e-42a699bd472c service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:09.845029 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b21a63-b420-43bd-927e-42a699bd472c service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:09.845273 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b21a63-b420-43bd-927e-42a699bd472c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:09.939874 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c212c7a5-c332-4323-b6da-c0fc58032e30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b639ad29baf4862b326e2d234d50452'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:09.962382 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c212c7a5-c332-4323-b6da-c0fc58032e30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b639ad29baf4862b326e2d234d50452'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:09.976900 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b21a63-b420-43bd-927e-42a699bd472c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:09.977710 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c212c7a5-c332-4323-b6da-c0fc58032e30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b639ad29baf4862b326e2d234d50452'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:09.978633 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b21a63-b420-43bd-927e-42a699bd472c service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:09.996043 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 951/1892] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:02:10.040048 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c212c7a5-c332-4323-b6da-c0fc58032e30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b639ad29baf4862b326e2d234d50452'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:10.040725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c212c7a5-c332-4323-b6da-c0fc58032e30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b639ad29baf4862b326e2d234d50452'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:10.052610 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 942/1893] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:02:09 2022] DELETE /identity/v3/users/1b639ad29baf4862b326e2d234d50452 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:10.054635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322df7a3-f39e-4a30-a5a7-4b71004cc467 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:10.055160 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322df7a3-f39e-4a30-a5a7-4b71004cc467 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:10.055444 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322df7a3-f39e-4a30-a5a7-4b71004cc467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:10.077895 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-322df7a3-f39e-4a30-a5a7-4b71004cc467 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:10.084672 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-322df7a3-f39e-4a30-a5a7-4b71004cc467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:10.160254 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 952/1894] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:10.173544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212c7a5-c332-4323-b6da-c0fc58032e30 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:10.177481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c212c7a5-c332-4323-b6da-c0fc58032e30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:10.251672 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212c7a5-c332-4323-b6da-c0fc58032e30 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:10.256651 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec2304-64f7-4aa8-903a-326d13f2ebb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:10.258245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec2304-64f7-4aa8-903a-326d13f2ebb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:10.258604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec2304-64f7-4aa8-903a-326d13f2ebb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:10.258974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec2304-64f7-4aa8-903a-326d13f2ebb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:10.275820 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ec2304-64f7-4aa8-903a-326d13f2ebb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:10.278250 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ec2304-64f7-4aa8-903a-326d13f2ebb2 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:10.291578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 943/1895] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:02:10.848661 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322df7a3-f39e-4a30-a5a7-4b71004cc467 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:10.849967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-322df7a3-f39e-4a30-a5a7-4b71004cc467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:10.863716 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322df7a3-f39e-4a30-a5a7-4b71004cc467 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:10.869765 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e27a8c-7d27-4a8c-9e5b-a0a571c69079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:10.870370 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e27a8c-7d27-4a8c-9e5b-a0a571c69079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:10.870647 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e27a8c-7d27-4a8c-9e5b-a0a571c69079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:10.870944 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e27a8c-7d27-4a8c-9e5b-a0a571c69079 admin admin] PATH_INFO: `/v3/projects/8a956c0898ec47c7a42706d27a72d94b` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:10.872116 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e27a8c-7d27-4a8c-9e5b-a0a571c69079 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a956c0898ec47c7a42706d27a72d94b)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:10.874027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e27a8c-7d27-4a8c-9e5b-a0a571c69079 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:11.024835 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c2e27a8c-7d27-4a8c-9e5b-a0a571c69079 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a956c0898ec47c7a42706d27a72d94b', 'request_id': 'req-c2e27a8c-7d27-4a8c-9e5b-a0a571c69079'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:11.165685 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c2e27a8c-7d27-4a8c-9e5b-a0a571c69079 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a956c0898ec47c7a42706d27a72d94b', 'request_id': 'req-c2e27a8c-7d27-4a8c-9e5b-a0a571c69079'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:11.177142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c2e27a8c-7d27-4a8c-9e5b-a0a571c69079 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a956c0898ec47c7a42706d27a72d94b', 'request_id': 'req-c2e27a8c-7d27-4a8c-9e5b-a0a571c69079'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:11.190947 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c2e27a8c-7d27-4a8c-9e5b-a0a571c69079 admin admin] The token cache is being invalidate because project 8a956c0898ec47c7a42706d27a72d94b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:11.191365 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c2e27a8c-7d27-4a8c-9e5b-a0a571c69079 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:11.192148 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c2e27a8c-7d27-4a8c-9e5b-a0a571c69079 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:11.198442 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec2304-64f7-4aa8-903a-326d13f2ebb2 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:11.204512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 953/1896] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:02:10 2022] DELETE /identity/v3/projects/8a956c0898ec47c7a42706d27a72d94b => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:11.210543 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec2304-64f7-4aa8-903a-326d13f2ebb2 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:11.261706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:11.280382 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366907fd-443f-4f95-8a98-5f2e68cd2bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:11.281105 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366907fd-443f-4f95-8a98-5f2e68cd2bdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:11.282687 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366907fd-443f-4f95-8a98-5f2e68cd2bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:11.282687 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366907fd-443f-4f95-8a98-5f2e68cd2bdc admin admin] PATH_INFO: `/v3/users/19dbfb73c5c54e7db9b9cc4f74a3cf7d` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:11.283031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366907fd-443f-4f95-8a98-5f2e68cd2bdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=19dbfb73c5c54e7db9b9cc4f74a3cf7d)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:11.285888 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366907fd-443f-4f95-8a98-5f2e68cd2bdc admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:11.574991 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-366907fd-443f-4f95-8a98-5f2e68cd2bdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19dbfb73c5c54e7db9b9cc4f74a3cf7d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:11.583257 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-366907fd-443f-4f95-8a98-5f2e68cd2bdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19dbfb73c5c54e7db9b9cc4f74a3cf7d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:11.606362 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-366907fd-443f-4f95-8a98-5f2e68cd2bdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19dbfb73c5c54e7db9b9cc4f74a3cf7d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:11.706998 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-366907fd-443f-4f95-8a98-5f2e68cd2bdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19dbfb73c5c54e7db9b9cc4f74a3cf7d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:11.708097 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-366907fd-443f-4f95-8a98-5f2e68cd2bdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19dbfb73c5c54e7db9b9cc4f74a3cf7d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:11.723383 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 944/1897] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:02:11 2022] DELETE /identity/v3/users/19dbfb73c5c54e7db9b9cc4f74a3cf7d => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:12.293773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e27a8c-7d27-4a8c-9e5b-a0a571c69079 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:12.295378 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c2e27a8c-7d27-4a8c-9e5b-a0a571c69079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:12.312764 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:12.322212 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e27a8c-7d27-4a8c-9e5b-a0a571c69079 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:12.416083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc84e83-ce11-4d02-b229-a07b868e20a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:12.416083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc84e83-ce11-4d02-b229-a07b868e20a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:12.416083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc84e83-ce11-4d02-b229-a07b868e20a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:12.416083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc84e83-ce11-4d02-b229-a07b868e20a3 admin admin] PATH_INFO: `/v3/projects/eecaa089cc2a46559a9b00db5621244a` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:12.417912 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc84e83-ce11-4d02-b229-a07b868e20a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eecaa089cc2a46559a9b00db5621244a)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:12.418955 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc84e83-ce11-4d02-b229-a07b868e20a3 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:12.557694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1bc84e83-ce11-4d02-b229-a07b868e20a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eecaa089cc2a46559a9b00db5621244a', 'request_id': 'req-1bc84e83-ce11-4d02-b229-a07b868e20a3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:12.640397 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1bc84e83-ce11-4d02-b229-a07b868e20a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eecaa089cc2a46559a9b00db5621244a', 'request_id': 'req-1bc84e83-ce11-4d02-b229-a07b868e20a3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:12.647634 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1bc84e83-ce11-4d02-b229-a07b868e20a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eecaa089cc2a46559a9b00db5621244a', 'request_id': 'req-1bc84e83-ce11-4d02-b229-a07b868e20a3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:12.663197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1bc84e83-ce11-4d02-b229-a07b868e20a3 admin admin] The token cache is being invalidate because project eecaa089cc2a46559a9b00db5621244a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:12.663560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1bc84e83-ce11-4d02-b229-a07b868e20a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:12.664298 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1bc84e83-ce11-4d02-b229-a07b868e20a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:12.667808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 954/1898] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:02:12 2022] DELETE /identity/v3/projects/eecaa089cc2a46559a9b00db5621244a => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:12.673504 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf18389-5539-4d12-a85c-f590395ec530 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:12.674557 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf18389-5539-4d12-a85c-f590395ec530 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:12.674557 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf18389-5539-4d12-a85c-f590395ec530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:12.706979 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-9cf18389-5539-4d12-a85c-f590395ec530 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:12.706979 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9cf18389-5539-4d12-a85c-f590395ec530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:12.805836 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 945/1899] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:12.815831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da6e25-49a8-4822-a782-15d1853a662f None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:12.816363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da6e25-49a8-4822-a782-15d1853a662f None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:12.816842 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da6e25-49a8-4822-a782-15d1853a662f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:12.876199 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-57da6e25-49a8-4822-a782-15d1853a662f None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:12.876199 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-57da6e25-49a8-4822-a782-15d1853a662f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:12.876199 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 955/1900] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:12.877018 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf18389-5539-4d12-a85c-f590395ec530 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:12.890039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf18389-5539-4d12-a85c-f590395ec530 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:12.895987 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4048716d-079f-440b-800d-fae7987c4665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:12.896745 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048716d-079f-440b-800d-fae7987c4665 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:12.897091 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048716d-079f-440b-800d-fae7987c4665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:12.897435 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048716d-079f-440b-800d-fae7987c4665 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:12.898330 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4048716d-079f-440b-800d-fae7987c4665 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:02:12.898732 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4048716d-079f-440b-800d-fae7987c4665 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:12.901101 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4048716d-079f-440b-800d-fae7987c4665 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:12.915508 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 946/1901] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:02:12.919554 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da6e25-49a8-4822-a782-15d1853a662f None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:12.933338 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da6e25-49a8-4822-a782-15d1853a662f None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:12.939646 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2ff6a-5e3c-40b1-8757-9cc99b802702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:12.941021 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2ff6a-5e3c-40b1-8757-9cc99b802702 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:12.941626 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2ff6a-5e3c-40b1-8757-9cc99b802702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:12.942064 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2ff6a-5e3c-40b1-8757-9cc99b802702 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:12.943399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec2ff6a-5e3c-40b1-8757-9cc99b802702 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:12.945374 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec2ff6a-5e3c-40b1-8757-9cc99b802702 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:13.098612 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 956/1902] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:02:12 2022] POST /identity/v3/projects => generated 379 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:02:13.102640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4048716d-079f-440b-800d-fae7987c4665 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:13.113661 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4048716d-079f-440b-800d-fae7987c4665 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:13.163024 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:13.175453 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402816a-d1fa-491d-91a7-b2b10c9e1681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:13.180703 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402816a-d1fa-491d-91a7-b2b10c9e1681 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:13.180703 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402816a-d1fa-491d-91a7-b2b10c9e1681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:13.180703 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402816a-d1fa-491d-91a7-b2b10c9e1681 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:13.180703 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e402816a-d1fa-491d-91a7-b2b10c9e1681 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:13.180703 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e402816a-d1fa-491d-91a7-b2b10c9e1681 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:13.323171 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 947/1903] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:02:13 2022] POST /identity/v3/users => generated 290 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:13.324467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2ff6a-5e3c-40b1-8757-9cc99b802702 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:13.335728 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2ff6a-5e3c-40b1-8757-9cc99b802702 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:13.340511 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458f09d-d0b3-4412-81e9-826d68bf2f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:13.341110 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458f09d-d0b3-4412-81e9-826d68bf2f06 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:13.341385 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458f09d-d0b3-4412-81e9-826d68bf2f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:13.341656 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458f09d-d0b3-4412-81e9-826d68bf2f06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:13.342291 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7458f09d-d0b3-4412-81e9-826d68bf2f06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:13.343931 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7458f09d-d0b3-4412-81e9-826d68bf2f06 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:13.355130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 957/1904] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:13.359218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402816a-d1fa-491d-91a7-b2b10c9e1681 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:13.370557 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402816a-d1fa-491d-91a7-b2b10c9e1681 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:13.381170 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a952987-d865-4c78-91dd-288ac857dc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:13.381781 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a952987-d865-4c78-91dd-288ac857dc6c admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:13.382054 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a952987-d865-4c78-91dd-288ac857dc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:13.382321 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a952987-d865-4c78-91dd-288ac857dc6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:13.383019 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a952987-d865-4c78-91dd-288ac857dc6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:13.384732 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a952987-d865-4c78-91dd-288ac857dc6c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:13.398411 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 948/1905] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:02:13.417825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458f09d-d0b3-4412-81e9-826d68bf2f06 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:13.428153 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458f09d-d0b3-4412-81e9-826d68bf2f06 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:13.433833 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c7022-58cd-4fc8-a0fd-1fee66b3a662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:13.434496 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c7022-58cd-4fc8-a0fd-1fee66b3a662 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:13.434825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c7022-58cd-4fc8-a0fd-1fee66b3a662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:13.435099 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c7022-58cd-4fc8-a0fd-1fee66b3a662 admin admin] PATH_INFO: `/v3/projects/84490ec3aa7b486b9a2dd7b112d6a128/users/10dd58029d754e75bae046fe97e2b03c/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:13.451812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002c7022-58cd-4fc8-a0fd-1fee66b3a662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84490ec3aa7b486b9a2dd7b112d6a128, user_id=10dd58029d754e75bae046fe97e2b03c, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:13.454900 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002c7022-58cd-4fc8-a0fd-1fee66b3a662 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:13.569373 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 958/1906] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:13 2022] PUT /identity/v3/projects/84490ec3aa7b486b9a2dd7b112d6a128/users/10dd58029d754e75bae046fe97e2b03c/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:13.581716 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b68106-d0a4-461d-afa5-cc91859a1ddd None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:13.581996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b68106-d0a4-461d-afa5-cc91859a1ddd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:13.582267 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b68106-d0a4-461d-afa5-cc91859a1ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:13.611220 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-70b68106-d0a4-461d-afa5-cc91859a1ddd None None] MFA Rules not processed for user `10dd58029d754e75bae046fe97e2b03c`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:13.612962 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-70b68106-d0a4-461d-afa5-cc91859a1ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:13.679682 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 949/1907] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:13.684204 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c7022-58cd-4fc8-a0fd-1fee66b3a662 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:13.717032 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:13.748891 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c7022-58cd-4fc8-a0fd-1fee66b3a662 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:13.753814 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53679c4-c3db-4ba9-a15c-223fe0200399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:13.757133 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53679c4-c3db-4ba9-a15c-223fe0200399 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:13.757133 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53679c4-c3db-4ba9-a15c-223fe0200399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:13.757133 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53679c4-c3db-4ba9-a15c-223fe0200399 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:13.757133 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53679c4-c3db-4ba9-a15c-223fe0200399 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:13.757644 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53679c4-c3db-4ba9-a15c-223fe0200399 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:13.882542 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 959/1908] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:02:13 2022] POST /identity/v3/projects => generated 381 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:13.886585 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b68106-d0a4-461d-afa5-cc91859a1ddd None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:13.897628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b68106-d0a4-461d-afa5-cc91859a1ddd None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:13.960310 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e854e0-d18b-4210-b9a7-fb68e8f2283a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:13.960829 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e854e0-d18b-4210-b9a7-fb68e8f2283a admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:13.961102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e854e0-d18b-4210-b9a7-fb68e8f2283a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:13.961367 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e854e0-d18b-4210-b9a7-fb68e8f2283a admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:13.962003 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e854e0-d18b-4210-b9a7-fb68e8f2283a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:13.963942 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e854e0-d18b-4210-b9a7-fb68e8f2283a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:14.125336 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 950/1909] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:02:13 2022] POST /identity/v3/users => generated 297 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:14.129719 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53679c4-c3db-4ba9-a15c-223fe0200399 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:14.139735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53679c4-c3db-4ba9-a15c-223fe0200399 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:14.144946 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38658556-1580-4198-aa3d-5580452af12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:14.145547 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38658556-1580-4198-aa3d-5580452af12f admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:14.145833 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38658556-1580-4198-aa3d-5580452af12f admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:14.146090 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38658556-1580-4198-aa3d-5580452af12f admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:14.146774 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38658556-1580-4198-aa3d-5580452af12f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:14.151714 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38658556-1580-4198-aa3d-5580452af12f admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:14.158074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 960/1910] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:02:14.163066 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e854e0-d18b-4210-b9a7-fb68e8f2283a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:14.173141 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e854e0-d18b-4210-b9a7-fb68e8f2283a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:14.178406 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d089852-a5a3-404b-8a4e-b2c00f393556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:14.179078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d089852-a5a3-404b-8a4e-b2c00f393556 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:14.179355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d089852-a5a3-404b-8a4e-b2c00f393556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:14.179623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d089852-a5a3-404b-8a4e-b2c00f393556 admin admin] PATH_INFO: `/v3/projects/caa8ce63435d40f7b5c68a1fd788a073/users/fcef7a6dbe334c3499ec0ea434ecfe7f/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:14.196290 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d089852-a5a3-404b-8a4e-b2c00f393556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caa8ce63435d40f7b5c68a1fd788a073, user_id=fcef7a6dbe334c3499ec0ea434ecfe7f, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:14.199195 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d089852-a5a3-404b-8a4e-b2c00f393556 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:14.256594 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 951/1911] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:14 2022] PUT /identity/v3/projects/caa8ce63435d40f7b5c68a1fd788a073/users/fcef7a6dbe334c3499ec0ea434ecfe7f/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:14.260405 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38658556-1580-4198-aa3d-5580452af12f admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:14.270970 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38658556-1580-4198-aa3d-5580452af12f admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:14.362146 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81386ca-da16-4ce4-a3b8-ebf8a45e8b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:14.362833 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81386ca-da16-4ce4-a3b8-ebf8a45e8b7d admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:14.363077 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81386ca-da16-4ce4-a3b8-ebf8a45e8b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:14.363384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81386ca-da16-4ce4-a3b8-ebf8a45e8b7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:14.364021 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81386ca-da16-4ce4-a3b8-ebf8a45e8b7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:14.365608 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81386ca-da16-4ce4-a3b8-ebf8a45e8b7d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:14.375685 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 961/1912] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:02:14.380212 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d089852-a5a3-404b-8a4e-b2c00f393556 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:14.390616 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d089852-a5a3-404b-8a4e-b2c00f393556 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:14.395998 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c722fb-31bb-450e-8a32-425b3b210eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:14.396678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c722fb-31bb-450e-8a32-425b3b210eae admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:14.396958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c722fb-31bb-450e-8a32-425b3b210eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:14.397228 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c722fb-31bb-450e-8a32-425b3b210eae admin admin] PATH_INFO: `/v3/projects/caa8ce63435d40f7b5c68a1fd788a073/users/fcef7a6dbe334c3499ec0ea434ecfe7f/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:14.399200 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c722fb-31bb-450e-8a32-425b3b210eae admin admin] RBAC: Authorizing `identity:create_grant(project_id=caa8ce63435d40f7b5c68a1fd788a073, user_id=fcef7a6dbe334c3499ec0ea434ecfe7f, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:14.401848 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c722fb-31bb-450e-8a32-425b3b210eae admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:14.412832 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 952/1913] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:14 2022] PUT /identity/v3/projects/caa8ce63435d40f7b5c68a1fd788a073/users/fcef7a6dbe334c3499ec0ea434ecfe7f/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:02:14.422945 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5d555-5e4b-4ac6-b179-588f50c2924b None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:14.423230 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5d555-5e4b-4ac6-b179-588f50c2924b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:14.423505 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5d555-5e4b-4ac6-b179-588f50c2924b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:14.452134 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-7eb5d555-5e4b-4ac6-b179-588f50c2924b None None] MFA Rules not processed for user `fcef7a6dbe334c3499ec0ea434ecfe7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:14.453355 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-7eb5d555-5e4b-4ac6-b179-588f50c2924b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:14.551327 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 962/1914] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:14 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:14.657414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c722fb-31bb-450e-8a32-425b3b210eae admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:14.659374 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-09c722fb-31bb-450e-8a32-425b3b210eae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:14.675823 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:14.722037 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c722fb-31bb-450e-8a32-425b3b210eae admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:14.726024 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1ab05-85b3-4c7a-8232-316a8c6ab650 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:14.729403 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1ab05-85b3-4c7a-8232-316a8c6ab650 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:14.729403 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1ab05-85b3-4c7a-8232-316a8c6ab650 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:14.729403 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1ab05-85b3-4c7a-8232-316a8c6ab650 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:14.735958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce1ab05-85b3-4c7a-8232-316a8c6ab650 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:14.737502 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce1ab05-85b3-4c7a-8232-316a8c6ab650 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:14.747025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 953/1915] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:14 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:02:15.851639 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5d555-5e4b-4ac6-b179-588f50c2924b None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:15.860758 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5d555-5e4b-4ac6-b179-588f50c2924b None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:15.866198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9304568-b58a-4f7d-9d59-10d046df1c19 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcef7a6dbe334c3499ec0ea434ecfe7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'caa8ce63435d40f7b5c68a1fd788a073', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:15.866853 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9304568-b58a-4f7d-9d59-10d046df1c19 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:15.867119 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9304568-b58a-4f7d-9d59-10d046df1c19 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:15.867392 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9304568-b58a-4f7d-9d59-10d046df1c19 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:15.868029 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9304568-b58a-4f7d-9d59-10d046df1c19 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:02:15.868328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9304568-b58a-4f7d-9d59-10d046df1c19 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:15.869945 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9304568-b58a-4f7d-9d59-10d046df1c19 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:15.881494 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 963/1916] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:02:15.885527 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1ab05-85b3-4c7a-8232-316a8c6ab650 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:15.895388 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1ab05-85b3-4c7a-8232-316a8c6ab650 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:15.899904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484cf04f-91c9-4d87-b0ff-af6169617a8f tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcef7a6dbe334c3499ec0ea434ecfe7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'caa8ce63435d40f7b5c68a1fd788a073', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:15.900489 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484cf04f-91c9-4d87-b0ff-af6169617a8f tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:15.900767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484cf04f-91c9-4d87-b0ff-af6169617a8f tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:15.901065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484cf04f-91c9-4d87-b0ff-af6169617a8f tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:15.901698 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484cf04f-91c9-4d87-b0ff-af6169617a8f tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:15.903702 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484cf04f-91c9-4d87-b0ff-af6169617a8f tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:15.993119 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 954/1917] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:02:15 2022] POST /identity/v3/projects => generated 353 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:02:15.994283 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9304568-b58a-4f7d-9d59-10d046df1c19 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:16.005773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9304568-b58a-4f7d-9d59-10d046df1c19 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:16.018749 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:16.079518 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6eb03-b235-4860-a1d5-66ee23c9e671 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcef7a6dbe334c3499ec0ea434ecfe7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'caa8ce63435d40f7b5c68a1fd788a073', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:16.081557 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6eb03-b235-4860-a1d5-66ee23c9e671 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:16.082079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6eb03-b235-4860-a1d5-66ee23c9e671 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:16.082424 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6eb03-b235-4860-a1d5-66ee23c9e671 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:16.083211 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6eb03-b235-4860-a1d5-66ee23c9e671 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:02:16.083596 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6eb03-b235-4860-a1d5-66ee23c9e671 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:16.087222 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6eb03-b235-4860-a1d5-66ee23c9e671 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:16.103703 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 964/1918] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:02:16.243664 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484cf04f-91c9-4d87-b0ff-af6169617a8f tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:16.254261 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484cf04f-91c9-4d87-b0ff-af6169617a8f tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:16.258810 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcabb2-8843-4245-933c-474a19a877a9 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcef7a6dbe334c3499ec0ea434ecfe7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'caa8ce63435d40f7b5c68a1fd788a073', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:16.259430 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcabb2-8843-4245-933c-474a19a877a9 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:16.259703 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcabb2-8843-4245-933c-474a19a877a9 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:16.259969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcabb2-8843-4245-933c-474a19a877a9 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] PATH_INFO: `/v3/projects/84f3f29f6d304b68a58286a9917b2a12` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:16.261158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbcabb2-8843-4245-933c-474a19a877a9 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] RBAC: Authorizing `identity:delete_project(project_id=84f3f29f6d304b68a58286a9917b2a12)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:16.263168 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbcabb2-8843-4245-933c-474a19a877a9 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:16.370688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bcbcabb2-8843-4245-933c-474a19a877a9 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84f3f29f6d304b68a58286a9917b2a12', 'request_id': 'req-bcbcabb2-8843-4245-933c-474a19a877a9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:16.377574 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bcbcabb2-8843-4245-933c-474a19a877a9 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84f3f29f6d304b68a58286a9917b2a12', 'request_id': 'req-bcbcabb2-8843-4245-933c-474a19a877a9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:16.385580 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bcbcabb2-8843-4245-933c-474a19a877a9 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84f3f29f6d304b68a58286a9917b2a12', 'request_id': 'req-bcbcabb2-8843-4245-933c-474a19a877a9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:16.471846 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bcbcabb2-8843-4245-933c-474a19a877a9 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] The token cache is being invalidate because project 84f3f29f6d304b68a58286a9917b2a12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:16.472233 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bcbcabb2-8843-4245-933c-474a19a877a9 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:16.472860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bcbcabb2-8843-4245-933c-474a19a877a9 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:16.474324 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 955/1919] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:02:16 2022] DELETE /identity/v3/projects/84f3f29f6d304b68a58286a9917b2a12 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:17.035595 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6eb03-b235-4860-a1d5-66ee23c9e671 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:17.037448 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0ea6eb03-b235-4860-a1d5-66ee23c9e671 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:17.103194 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6eb03-b235-4860-a1d5-66ee23c9e671 tempest-VolumeQuotaClassesTest-1448275724 tempest-VolumeQuotaClassesTest-1448275724-project-admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:17.161896 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202133b-9840-4b2c-9d04-6159f6dc289e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:17.161896 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202133b-9840-4b2c-9d04-6159f6dc289e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:17.161896 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202133b-9840-4b2c-9d04-6159f6dc289e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:17.161896 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202133b-9840-4b2c-9d04-6159f6dc289e admin admin] PATH_INFO: `/v3/users/10dd58029d754e75bae046fe97e2b03c` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:17.162976 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9202133b-9840-4b2c-9d04-6159f6dc289e admin admin] RBAC: Authorizing `identity:delete_user(user_id=10dd58029d754e75bae046fe97e2b03c)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:17.164891 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9202133b-9840-4b2c-9d04-6159f6dc289e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:17.374271 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-9202133b-9840-4b2c-9d04-6159f6dc289e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10dd58029d754e75bae046fe97e2b03c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:17.379568 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:17.386298 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-9202133b-9840-4b2c-9d04-6159f6dc289e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10dd58029d754e75bae046fe97e2b03c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:17.393846 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-9202133b-9840-4b2c-9d04-6159f6dc289e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10dd58029d754e75bae046fe97e2b03c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:17.396627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-9202133b-9840-4b2c-9d04-6159f6dc289e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10dd58029d754e75bae046fe97e2b03c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:17.467870 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-9202133b-9840-4b2c-9d04-6159f6dc289e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10dd58029d754e75bae046fe97e2b03c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:17.477087 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 965/1920] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:02:17 2022] DELETE /identity/v3/users/10dd58029d754e75bae046fe97e2b03c => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:02:17.482606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3380d-4877-4ba2-af73-7e6448b12765 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:17.483578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3380d-4877-4ba2-af73-7e6448b12765 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:17.484017 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3380d-4877-4ba2-af73-7e6448b12765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:17.512334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d7b3380d-4877-4ba2-af73-7e6448b12765 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:17.513497 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d7b3380d-4877-4ba2-af73-7e6448b12765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:17.575087 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 956/1921] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:17.584891 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202133b-9840-4b2c-9d04-6159f6dc289e admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:17.586306 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9202133b-9840-4b2c-9d04-6159f6dc289e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:17.648173 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202133b-9840-4b2c-9d04-6159f6dc289e admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:17.651744 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a64ad4-3450-4e11-8326-695d27f2ca73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:17.652341 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a64ad4-3450-4e11-8326-695d27f2ca73 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:17.652618 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a64ad4-3450-4e11-8326-695d27f2ca73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:17.652885 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a64ad4-3450-4e11-8326-695d27f2ca73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:17.661990 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a64ad4-3450-4e11-8326-695d27f2ca73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:17.663497 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a64ad4-3450-4e11-8326-695d27f2ca73 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:17.672830 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 966/1922] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:02:18.108993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3380d-4877-4ba2-af73-7e6448b12765 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:18.112089 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d7b3380d-4877-4ba2-af73-7e6448b12765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:18.126913 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3380d-4877-4ba2-af73-7e6448b12765 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:18.132069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4db0a-b6f3-4a11-9b62-04cf32b5537c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:18.132711 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4db0a-b6f3-4a11-9b62-04cf32b5537c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:18.133069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4db0a-b6f3-4a11-9b62-04cf32b5537c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:18.133247 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4db0a-b6f3-4a11-9b62-04cf32b5537c admin admin] PATH_INFO: `/v3/projects/84490ec3aa7b486b9a2dd7b112d6a128` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:18.134362 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f4db0a-b6f3-4a11-9b62-04cf32b5537c admin admin] RBAC: Authorizing `identity:delete_project(project_id=84490ec3aa7b486b9a2dd7b112d6a128)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:18.140994 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f4db0a-b6f3-4a11-9b62-04cf32b5537c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:18.148128 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:18.257021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-97f4db0a-b6f3-4a11-9b62-04cf32b5537c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84490ec3aa7b486b9a2dd7b112d6a128', 'request_id': 'req-97f4db0a-b6f3-4a11-9b62-04cf32b5537c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:18.263108 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-97f4db0a-b6f3-4a11-9b62-04cf32b5537c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84490ec3aa7b486b9a2dd7b112d6a128', 'request_id': 'req-97f4db0a-b6f3-4a11-9b62-04cf32b5537c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:18.270148 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-97f4db0a-b6f3-4a11-9b62-04cf32b5537c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84490ec3aa7b486b9a2dd7b112d6a128', 'request_id': 'req-97f4db0a-b6f3-4a11-9b62-04cf32b5537c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:18.348415 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-97f4db0a-b6f3-4a11-9b62-04cf32b5537c admin admin] The token cache is being invalidate because project 84490ec3aa7b486b9a2dd7b112d6a128 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:18.348415 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-97f4db0a-b6f3-4a11-9b62-04cf32b5537c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:18.349067 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-97f4db0a-b6f3-4a11-9b62-04cf32b5537c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:18.350810 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 957/1923] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:02:18 2022] DELETE /identity/v3/projects/84490ec3aa7b486b9a2dd7b112d6a128 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:02:18.357246 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a64ad4-3450-4e11-8326-695d27f2ca73 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:18.368290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a64ad4-3450-4e11-8326-695d27f2ca73 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:18.429555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed692748-3bd6-4380-a4c6-c94045348d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:18.430176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed692748-3bd6-4380-a4c6-c94045348d8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:18.435482 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed692748-3bd6-4380-a4c6-c94045348d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:18.435482 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed692748-3bd6-4380-a4c6-c94045348d8c admin admin] PATH_INFO: `/v3/users/fcef7a6dbe334c3499ec0ea434ecfe7f` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:18.435482 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed692748-3bd6-4380-a4c6-c94045348d8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcef7a6dbe334c3499ec0ea434ecfe7f)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:18.435482 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed692748-3bd6-4380-a4c6-c94045348d8c admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:18.645179 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ed692748-3bd6-4380-a4c6-c94045348d8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcef7a6dbe334c3499ec0ea434ecfe7f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:18.658202 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ed692748-3bd6-4380-a4c6-c94045348d8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcef7a6dbe334c3499ec0ea434ecfe7f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:18.664985 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ed692748-3bd6-4380-a4c6-c94045348d8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcef7a6dbe334c3499ec0ea434ecfe7f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:18.666406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ed692748-3bd6-4380-a4c6-c94045348d8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcef7a6dbe334c3499ec0ea434ecfe7f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:18.668291 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:18.738104 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ed692748-3bd6-4380-a4c6-c94045348d8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcef7a6dbe334c3499ec0ea434ecfe7f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:18.744866 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 967/1924] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:02:18 2022] DELETE /identity/v3/users/fcef7a6dbe334c3499ec0ea434ecfe7f => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:02:19.153099 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4db0a-b6f3-4a11-9b62-04cf32b5537c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:19.154639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-97f4db0a-b6f3-4a11-9b62-04cf32b5537c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:19.168188 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4db0a-b6f3-4a11-9b62-04cf32b5537c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:19.231336 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5b677-381d-4a31-8111-e1a2d74b9731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:19.231995 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5b677-381d-4a31-8111-e1a2d74b9731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:19.232273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5b677-381d-4a31-8111-e1a2d74b9731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:19.232542 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5b677-381d-4a31-8111-e1a2d74b9731 admin admin] PATH_INFO: `/v3/projects/caa8ce63435d40f7b5c68a1fd788a073` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:19.233627 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd5b677-381d-4a31-8111-e1a2d74b9731 admin admin] RBAC: Authorizing `identity:delete_project(project_id=caa8ce63435d40f7b5c68a1fd788a073)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:19.236005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd5b677-381d-4a31-8111-e1a2d74b9731 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:19.398962 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6fd5b677-381d-4a31-8111-e1a2d74b9731 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caa8ce63435d40f7b5c68a1fd788a073', 'request_id': 'req-6fd5b677-381d-4a31-8111-e1a2d74b9731'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:19.407099 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6fd5b677-381d-4a31-8111-e1a2d74b9731 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caa8ce63435d40f7b5c68a1fd788a073', 'request_id': 'req-6fd5b677-381d-4a31-8111-e1a2d74b9731'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:19.417961 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6fd5b677-381d-4a31-8111-e1a2d74b9731 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caa8ce63435d40f7b5c68a1fd788a073', 'request_id': 'req-6fd5b677-381d-4a31-8111-e1a2d74b9731'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:19.523301 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6fd5b677-381d-4a31-8111-e1a2d74b9731 admin admin] The token cache is being invalidate because project caa8ce63435d40f7b5c68a1fd788a073 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:19.523602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6fd5b677-381d-4a31-8111-e1a2d74b9731 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:19.524240 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6fd5b677-381d-4a31-8111-e1a2d74b9731 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:19.525864 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 958/1925] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:02:19 2022] DELETE /identity/v3/projects/caa8ce63435d40f7b5c68a1fd788a073 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:19.530688 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b8373-8135-4830-969f-18b129e92147 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:19.531174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b8373-8135-4830-969f-18b129e92147 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:19.531271 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b8373-8135-4830-969f-18b129e92147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:19.555131 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-684b8373-8135-4830-969f-18b129e92147 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:19.555131 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-684b8373-8135-4830-969f-18b129e92147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:19.621928 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 968/1926] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:19.629964 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50d009-4a3a-4d9b-ab90-f767213b3268 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:19.630273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50d009-4a3a-4d9b-ab90-f767213b3268 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:19.630549 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50d009-4a3a-4d9b-ab90-f767213b3268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:19.650583 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-bd50d009-4a3a-4d9b-ab90-f767213b3268 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:19.651817 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-bd50d009-4a3a-4d9b-ab90-f767213b3268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:19.665022 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 959/1927] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:19.669455 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b8373-8135-4830-969f-18b129e92147 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:19.679600 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b8373-8135-4830-969f-18b129e92147 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:19.684401 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5877fa-d172-4d2c-8e8b-6e0459f48b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:19.685114 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5877fa-d172-4d2c-8e8b-6e0459f48b7c admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:19.685393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5877fa-d172-4d2c-8e8b-6e0459f48b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:19.685669 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5877fa-d172-4d2c-8e8b-6e0459f48b7c admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:19.686311 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5877fa-d172-4d2c-8e8b-6e0459f48b7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:02:19.686638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5877fa-d172-4d2c-8e8b-6e0459f48b7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:19.688283 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5877fa-d172-4d2c-8e8b-6e0459f48b7c admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:19.711276 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 969/1928] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:02:19.711474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50d009-4a3a-4d9b-ab90-f767213b3268 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:19.718761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50d009-4a3a-4d9b-ab90-f767213b3268 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:19.727752 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4521c-3637-490b-869f-9f7a3ba11e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:19.728448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4521c-3637-490b-869f-9f7a3ba11e79 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:19.728690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4521c-3637-490b-869f-9f7a3ba11e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:19.728969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4521c-3637-490b-869f-9f7a3ba11e79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:19.729606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad4521c-3637-490b-869f-9f7a3ba11e79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:19.731590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad4521c-3637-490b-869f-9f7a3ba11e79 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:19.812152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:19.824727 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5877fa-d172-4d2c-8e8b-6e0459f48b7c admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:19.825233 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 960/1929] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:02:19 2022] POST /identity/v3/projects => generated 387 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:19.835267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5877fa-d172-4d2c-8e8b-6e0459f48b7c admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:19.893550 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d54eb0-ccde-4396-97ca-370ee2762433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:19.894165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d54eb0-ccde-4396-97ca-370ee2762433 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:19.894489 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d54eb0-ccde-4396-97ca-370ee2762433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:19.895103 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d54eb0-ccde-4396-97ca-370ee2762433 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:19.899467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d54eb0-ccde-4396-97ca-370ee2762433 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:19.899467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d54eb0-ccde-4396-97ca-370ee2762433 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:20.009153 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 970/1930] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:02:19 2022] POST /identity/v3/users => generated 294 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:02:20.012783 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4521c-3637-490b-869f-9f7a3ba11e79 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:20.029521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4521c-3637-490b-869f-9f7a3ba11e79 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:20.035365 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427400c7-e4d2-47fd-bb83-dd2efd03fd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:20.035909 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427400c7-e4d2-47fd-bb83-dd2efd03fd8a admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:20.036189 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427400c7-e4d2-47fd-bb83-dd2efd03fd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:20.036459 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427400c7-e4d2-47fd-bb83-dd2efd03fd8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:20.037138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427400c7-e4d2-47fd-bb83-dd2efd03fd8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:20.038847 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427400c7-e4d2-47fd-bb83-dd2efd03fd8a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:20.050740 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 961/1931] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:20.055320 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d54eb0-ccde-4396-97ca-370ee2762433 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:20.066860 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d54eb0-ccde-4396-97ca-370ee2762433 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:20.072151 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf123c07-c789-4da7-ba90-ac50a5ca0950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:20.072795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf123c07-c789-4da7-ba90-ac50a5ca0950 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:20.073072 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf123c07-c789-4da7-ba90-ac50a5ca0950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:20.073342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf123c07-c789-4da7-ba90-ac50a5ca0950 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:20.074012 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf123c07-c789-4da7-ba90-ac50a5ca0950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:20.075699 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf123c07-c789-4da7-ba90-ac50a5ca0950 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:20.086633 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 971/1932] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:20.090955 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427400c7-e4d2-47fd-bb83-dd2efd03fd8a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:20.101387 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427400c7-e4d2-47fd-bb83-dd2efd03fd8a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:20.105938 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c6ea2-d183-4180-82d8-12afc4de6b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:20.106578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c6ea2-d183-4180-82d8-12afc4de6b8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:20.106895 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c6ea2-d183-4180-82d8-12afc4de6b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:20.107168 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c6ea2-d183-4180-82d8-12afc4de6b8c admin admin] PATH_INFO: `/v3/projects/3bb6658f278f43b69b9d1a741fc2f82e/users/5f26791a55834f7cacbef8c281e7deec/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:20.121946 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c6ea2-d183-4180-82d8-12afc4de6b8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bb6658f278f43b69b9d1a741fc2f82e, user_id=5f26791a55834f7cacbef8c281e7deec, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:20.124640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c6ea2-d183-4180-82d8-12afc4de6b8c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:20.233194 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 962/1933] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:20 2022] PUT /identity/v3/projects/3bb6658f278f43b69b9d1a741fc2f82e/users/5f26791a55834f7cacbef8c281e7deec/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:20.245164 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af41d57-a2d7-4715-a48f-081cc187603b None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:20.245447 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af41d57-a2d7-4715-a48f-081cc187603b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:20.245714 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af41d57-a2d7-4715-a48f-081cc187603b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:20.274491 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-2af41d57-a2d7-4715-a48f-081cc187603b None None] MFA Rules not processed for user `5f26791a55834f7cacbef8c281e7deec`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:20.275737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2af41d57-a2d7-4715-a48f-081cc187603b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:20.324328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:20.342582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 972/1934] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:20.348164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c6ea2-d183-4180-82d8-12afc4de6b8c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:20.420243 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c6ea2-d183-4180-82d8-12afc4de6b8c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:20.426401 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9881d-fda8-4ffa-b00d-659a750edb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:20.427193 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9881d-fda8-4ffa-b00d-659a750edb1e admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:20.427570 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9881d-fda8-4ffa-b00d-659a750edb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:20.427928 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9881d-fda8-4ffa-b00d-659a750edb1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:20.428679 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9881d-fda8-4ffa-b00d-659a750edb1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:20.431294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9881d-fda8-4ffa-b00d-659a750edb1e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:20.509122 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 963/1935] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:02:20 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:20.513343 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af41d57-a2d7-4715-a48f-081cc187603b None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:20.526369 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af41d57-a2d7-4715-a48f-081cc187603b None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:20.584102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fa266-1060-40b9-a11e-618f5aa1332b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:20.584842 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fa266-1060-40b9-a11e-618f5aa1332b admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:20.585087 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fa266-1060-40b9-a11e-618f5aa1332b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:20.585359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fa266-1060-40b9-a11e-618f5aa1332b admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:20.585991 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1fa266-1060-40b9-a11e-618f5aa1332b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:20.587959 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1fa266-1060-40b9-a11e-618f5aa1332b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:20.614887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9881d-fda8-4ffa-b00d-659a750edb1e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:20.616667 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0fe9881d-fda8-4ffa-b00d-659a750edb1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:20.629779 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9881d-fda8-4ffa-b00d-659a750edb1e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:20.634542 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd51e5-55eb-4d00-b625-590b2766ef36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:20.635206 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd51e5-55eb-4d00-b625-590b2766ef36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:20.635514 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd51e5-55eb-4d00-b625-590b2766ef36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:20.635793 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd51e5-55eb-4d00-b625-590b2766ef36 admin admin] PATH_INFO: `/v3/users/a6c4688db12d420492c64ce3c4b29828` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:20.636925 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd51e5-55eb-4d00-b625-590b2766ef36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6c4688db12d420492c64ce3c4b29828)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:20.638952 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd51e5-55eb-4d00-b625-590b2766ef36 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:20.687423 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 973/1936] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:02:20 2022] POST /identity/v3/users => generated 294 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:20.688820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fa266-1060-40b9-a11e-618f5aa1332b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:20.698565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fa266-1060-40b9-a11e-618f5aa1332b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:20.703174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346b009-dae3-459c-bad2-67b9de089685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:20.703746 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346b009-dae3-459c-bad2-67b9de089685 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:20.704020 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346b009-dae3-459c-bad2-67b9de089685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:20.704287 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346b009-dae3-459c-bad2-67b9de089685 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:20.705449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c346b009-dae3-459c-bad2-67b9de089685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:20.707229 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c346b009-dae3-459c-bad2-67b9de089685 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:20.717129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 974/1937] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:02:20.721552 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346b009-dae3-459c-bad2-67b9de089685 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:20.731302 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346b009-dae3-459c-bad2-67b9de089685 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:20.735927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7c327-2ea2-460c-91d8-8df435665b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:20.736699 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7c327-2ea2-460c-91d8-8df435665b96 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:20.737198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7c327-2ea2-460c-91d8-8df435665b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:20.737517 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7c327-2ea2-460c-91d8-8df435665b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:20.738154 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a7c327-2ea2-460c-91d8-8df435665b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:20.739781 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a7c327-2ea2-460c-91d8-8df435665b96 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:20.756821 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 975/1938] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:02:20.757849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7c327-2ea2-460c-91d8-8df435665b96 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:20.769015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7c327-2ea2-460c-91d8-8df435665b96 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:20.773683 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee48fb9-b83c-4807-9e2e-e04233d6dc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:20.774347 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee48fb9-b83c-4807-9e2e-e04233d6dc12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:20.775594 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee48fb9-b83c-4807-9e2e-e04233d6dc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:20.775594 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee48fb9-b83c-4807-9e2e-e04233d6dc12 admin admin] PATH_INFO: `/v3/projects/74567b6cf5f3460eac8de09251dc358c/users/1ee9866bb84745cbbf515acf5e814113/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:20.789987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee48fb9-b83c-4807-9e2e-e04233d6dc12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74567b6cf5f3460eac8de09251dc358c, user_id=1ee9866bb84745cbbf515acf5e814113, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:20.792721 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee48fb9-b83c-4807-9e2e-e04233d6dc12 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:20.888789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-aadd51e5-55eb-4d00-b625-590b2766ef36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6c4688db12d420492c64ce3c4b29828'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:20.897491 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-aadd51e5-55eb-4d00-b625-590b2766ef36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6c4688db12d420492c64ce3c4b29828'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:20.914749 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 976/1939] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:20 2022] PUT /identity/v3/projects/74567b6cf5f3460eac8de09251dc358c/users/1ee9866bb84745cbbf515acf5e814113/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:02:20.914974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-aadd51e5-55eb-4d00-b625-590b2766ef36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6c4688db12d420492c64ce3c4b29828'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:20.918515 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528fac3-24e5-4321-838f-f10336ee9528 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:20.918901 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528fac3-24e5-4321-838f-f10336ee9528 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:20.919113 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528fac3-24e5-4321-838f-f10336ee9528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:20.947602 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-e528fac3-24e5-4321-838f-f10336ee9528 None None] MFA Rules not processed for user `1ee9866bb84745cbbf515acf5e814113`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:20.948807 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e528fac3-24e5-4321-838f-f10336ee9528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:20.978522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-aadd51e5-55eb-4d00-b625-590b2766ef36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6c4688db12d420492c64ce3c4b29828'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:20.980054 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-aadd51e5-55eb-4d00-b625-590b2766ef36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6c4688db12d420492c64ce3c4b29828'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:20.998766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 964/1940] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:02:20 2022] DELETE /identity/v3/users/a6c4688db12d420492c64ce3c4b29828 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:02:21.007559 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31af18-73a9-426b-8def-3c632eae38a8 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:21.016460 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31af18-73a9-426b-8def-3c632eae38a8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:21.016460 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31af18-73a9-426b-8def-3c632eae38a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:21.029708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 977/1941] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:02:21.034439 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528fac3-24e5-4321-838f-f10336ee9528 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:21.035900 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e528fac3-24e5-4321-838f-f10336ee9528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:21.041992 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ee31af18-73a9-426b-8def-3c632eae38a8 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:21.043373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ee31af18-73a9-426b-8def-3c632eae38a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:21.062108 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:21.066284 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:21.123966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528fac3-24e5-4321-838f-f10336ee9528 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:21.126968 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 965/1942] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:21.132249 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31af18-73a9-426b-8def-3c632eae38a8 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:21.134154 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ee31af18-73a9-426b-8def-3c632eae38a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:21.136641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6cf75-3732-44f9-b5b6-2a68a26731e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:21.137335 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6cf75-3732-44f9-b5b6-2a68a26731e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:21.137616 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6cf75-3732-44f9-b5b6-2a68a26731e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:21.137880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6cf75-3732-44f9-b5b6-2a68a26731e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:21.138525 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6cf75-3732-44f9-b5b6-2a68a26731e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:21.140749 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6cf75-3732-44f9-b5b6-2a68a26731e0 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:21.200485 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31af18-73a9-426b-8def-3c632eae38a8 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:21.205931 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade65ac-3631-41e0-b848-0d14021f2ff4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:21.206448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade65ac-3631-41e0-b848-0d14021f2ff4 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:21.206758 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade65ac-3631-41e0-b848-0d14021f2ff4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:21.207031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade65ac-3631-41e0-b848-0d14021f2ff4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:21.216919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade65ac-3631-41e0-b848-0d14021f2ff4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:21.218439 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade65ac-3631-41e0-b848-0d14021f2ff4 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:21.243178 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 966/1943] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:02:21.264128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 978/1944] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:02:21 2022] POST /identity/v3/projects => generated 385 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:21.268371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade65ac-3631-41e0-b848-0d14021f2ff4 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:21.287180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade65ac-3631-41e0-b848-0d14021f2ff4 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:21.357148 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04795c9b-6959-4e70-847d-f211fcd2d763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:21.357148 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04795c9b-6959-4e70-847d-f211fcd2d763 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:21.357148 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04795c9b-6959-4e70-847d-f211fcd2d763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:21.357148 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04795c9b-6959-4e70-847d-f211fcd2d763 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:21.357583 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04795c9b-6959-4e70-847d-f211fcd2d763 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:21.359614 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04795c9b-6959-4e70-847d-f211fcd2d763 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:21.461313 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 967/1945] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:02:21 2022] POST /identity/v3/users => generated 299 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:21.465560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6cf75-3732-44f9-b5b6-2a68a26731e0 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:21.476295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6cf75-3732-44f9-b5b6-2a68a26731e0 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:21.480858 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74336d47-9d1e-431e-a672-16bf9ff8857f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:21.481512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74336d47-9d1e-431e-a672-16bf9ff8857f admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:21.481740 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74336d47-9d1e-431e-a672-16bf9ff8857f admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:21.482015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74336d47-9d1e-431e-a672-16bf9ff8857f admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:21.482680 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74336d47-9d1e-431e-a672-16bf9ff8857f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:21.484294 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74336d47-9d1e-431e-a672-16bf9ff8857f admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:21.497318 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 979/1946] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:02:21.500119 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04795c9b-6959-4e70-847d-f211fcd2d763 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:21.512923 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04795c9b-6959-4e70-847d-f211fcd2d763 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:21.517909 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a467a399-d243-4600-b4b2-22871d767c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:21.518569 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467a399-d243-4600-b4b2-22871d767c02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:21.519901 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467a399-d243-4600-b4b2-22871d767c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:21.519901 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467a399-d243-4600-b4b2-22871d767c02 admin admin] PATH_INFO: `/v3/projects/a7245f20608948f7b9c383d9271cce2b/users/29399bab1ac347b4a546bb84eca6d05a/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:21.536014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a467a399-d243-4600-b4b2-22871d767c02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7245f20608948f7b9c383d9271cce2b, user_id=29399bab1ac347b4a546bb84eca6d05a, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:21.540051 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a467a399-d243-4600-b4b2-22871d767c02 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:21.654006 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 968/1947] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:21 2022] PUT /identity/v3/projects/a7245f20608948f7b9c383d9271cce2b/users/29399bab1ac347b4a546bb84eca6d05a/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:02:21.655266 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74336d47-9d1e-431e-a672-16bf9ff8857f admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:21.669066 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74336d47-9d1e-431e-a672-16bf9ff8857f admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:21.728541 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a467a399-d243-4600-b4b2-22871d767c02 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:21.731119 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a467a399-d243-4600-b4b2-22871d767c02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:21.762431 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e86e6-3072-4d91-8b0c-ddc6cf2e786c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:21.763337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e86e6-3072-4d91-8b0c-ddc6cf2e786c admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:21.763759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e86e6-3072-4d91-8b0c-ddc6cf2e786c admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:21.764173 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e86e6-3072-4d91-8b0c-ddc6cf2e786c admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:21.765039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04e86e6-3072-4d91-8b0c-ddc6cf2e786c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:21.768730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04e86e6-3072-4d91-8b0c-ddc6cf2e786c admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:21.786417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 980/1948] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:02:21.787399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e86e6-3072-4d91-8b0c-ddc6cf2e786c admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:21.801295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e86e6-3072-4d91-8b0c-ddc6cf2e786c admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:21.808537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469faa9-21da-4329-bbd1-39ec7ea21383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:21.809388 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469faa9-21da-4329-bbd1-39ec7ea21383 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:21.809677 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469faa9-21da-4329-bbd1-39ec7ea21383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:21.811424 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469faa9-21da-4329-bbd1-39ec7ea21383 admin admin] PATH_INFO: `/v3/projects/a7245f20608948f7b9c383d9271cce2b/users/29399bab1ac347b4a546bb84eca6d05a/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:21.812576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3469faa9-21da-4329-bbd1-39ec7ea21383 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7245f20608948f7b9c383d9271cce2b, user_id=29399bab1ac347b4a546bb84eca6d05a, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:21.816220 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3469faa9-21da-4329-bbd1-39ec7ea21383 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:21.841350 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 981/1949] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:21 2022] PUT /identity/v3/projects/a7245f20608948f7b9c383d9271cce2b/users/29399bab1ac347b4a546bb84eca6d05a/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:02:21.845078 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba152a-d74c-41d4-8e7f-3254b5e4d6d5 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:21.845380 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba152a-d74c-41d4-8e7f-3254b5e4d6d5 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:21.845715 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba152a-d74c-41d4-8e7f-3254b5e4d6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:21.849922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a467a399-d243-4600-b4b2-22871d767c02 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:21.883885 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-6cba152a-d74c-41d4-8e7f-3254b5e4d6d5 None None] MFA Rules not processed for user `29399bab1ac347b4a546bb84eca6d05a`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:21.885362 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6cba152a-d74c-41d4-8e7f-3254b5e4d6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:21.951864 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d4ea8-66b6-433d-9779-01d927729507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:21.952698 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d4ea8-66b6-433d-9779-01d927729507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:21.953138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d4ea8-66b6-433d-9779-01d927729507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:21.953536 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d4ea8-66b6-433d-9779-01d927729507 admin admin] PATH_INFO: `/v3/projects/ce035fd37c1549b7a3edaf65fa2f0f30` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:21.955271 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d4ea8-66b6-433d-9779-01d927729507 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce035fd37c1549b7a3edaf65fa2f0f30)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:21.959229 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d4ea8-66b6-433d-9779-01d927729507 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:21.981846 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:22.001467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 982/1950] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:21 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:02:22.016643 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba152a-d74c-41d4-8e7f-3254b5e4d6d5 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:22.019131 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6cba152a-d74c-41d4-8e7f-3254b5e4d6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:22.028529 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:22.093819 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba152a-d74c-41d4-8e7f-3254b5e4d6d5 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:22.098381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b59f3d-2037-40b5-9654-a3ff926ec3fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:22.099080 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b59f3d-2037-40b5-9654-a3ff926ec3fe service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:22.099357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b59f3d-2037-40b5-9654-a3ff926ec3fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:22.099657 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b59f3d-2037-40b5-9654-a3ff926ec3fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:22.114109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b59f3d-2037-40b5-9654-a3ff926ec3fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:22.115579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b59f3d-2037-40b5-9654-a3ff926ec3fe service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:22.127535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 983/1951] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:22 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:02:22.144466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-632d4ea8-66b6-433d-9779-01d927729507 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce035fd37c1549b7a3edaf65fa2f0f30', 'request_id': 'req-632d4ea8-66b6-433d-9779-01d927729507'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:22.153507 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-632d4ea8-66b6-433d-9779-01d927729507 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce035fd37c1549b7a3edaf65fa2f0f30', 'request_id': 'req-632d4ea8-66b6-433d-9779-01d927729507'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:22.160229 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-632d4ea8-66b6-433d-9779-01d927729507 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce035fd37c1549b7a3edaf65fa2f0f30', 'request_id': 'req-632d4ea8-66b6-433d-9779-01d927729507'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:22.272622 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-632d4ea8-66b6-433d-9779-01d927729507 admin admin] The token cache is being invalidate because project ce035fd37c1549b7a3edaf65fa2f0f30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:22.273682 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-632d4ea8-66b6-433d-9779-01d927729507 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:22.276828 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-632d4ea8-66b6-433d-9779-01d927729507 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:22.279634 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 969/1952] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:02:21 2022] DELETE /identity/v3/projects/ce035fd37c1549b7a3edaf65fa2f0f30 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:22.285760 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b59f3d-2037-40b5-9654-a3ff926ec3fe service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:22.317180 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b59f3d-2037-40b5-9654-a3ff926ec3fe service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:22.423382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d131b12-8e57-4436-9b96-8e98d3ce4297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:22.425214 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d131b12-8e57-4436-9b96-8e98d3ce4297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:22.425592 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d131b12-8e57-4436-9b96-8e98d3ce4297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:22.426076 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d131b12-8e57-4436-9b96-8e98d3ce4297 admin admin] PATH_INFO: `/v3/users/e5f59a72ca3c40cba8c14c9193162f7d` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:22.427496 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d131b12-8e57-4436-9b96-8e98d3ce4297 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5f59a72ca3c40cba8c14c9193162f7d)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:22.436832 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d131b12-8e57-4436-9b96-8e98d3ce4297 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:22.691371 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-6d131b12-8e57-4436-9b96-8e98d3ce4297 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5f59a72ca3c40cba8c14c9193162f7d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:22.713142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-6d131b12-8e57-4436-9b96-8e98d3ce4297 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5f59a72ca3c40cba8c14c9193162f7d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:22.716636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-6d131b12-8e57-4436-9b96-8e98d3ce4297 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5f59a72ca3c40cba8c14c9193162f7d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:22.717372 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-6d131b12-8e57-4436-9b96-8e98d3ce4297 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5f59a72ca3c40cba8c14c9193162f7d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:22.816440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-6d131b12-8e57-4436-9b96-8e98d3ce4297 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5f59a72ca3c40cba8c14c9193162f7d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:22.825061 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 984/1953] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:02:22 2022] DELETE /identity/v3/users/e5f59a72ca3c40cba8c14c9193162f7d => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:02:23.288241 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d4ea8-66b6-433d-9779-01d927729507 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:23.291597 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-632d4ea8-66b6-433d-9779-01d927729507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:23.314790 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d4ea8-66b6-433d-9779-01d927729507 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:23.384911 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d131b12-8e57-4436-9b96-8e98d3ce4297 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:23.386447 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6d131b12-8e57-4436-9b96-8e98d3ce4297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:23.394190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e1377-3443-4296-a0d7-4484a4083053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:23.394869 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e1377-3443-4296-a0d7-4484a4083053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:23.395145 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e1377-3443-4296-a0d7-4484a4083053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:23.395411 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e1377-3443-4296-a0d7-4484a4083053 admin admin] PATH_INFO: `/v3/projects/beee0e3d927642589c064aac5e697ff4` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:23.396732 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57e1377-3443-4296-a0d7-4484a4083053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=beee0e3d927642589c064aac5e697ff4)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:23.398650 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57e1377-3443-4296-a0d7-4484a4083053 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:23.411537 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:23.429261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:23.459734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d131b12-8e57-4436-9b96-8e98d3ce4297 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:23.463646 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c5371-8d8d-4913-b30a-ec19f24998ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:23.468392 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c5371-8d8d-4913-b30a-ec19f24998ec service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:23.468392 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c5371-8d8d-4913-b30a-ec19f24998ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:23.468392 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c5371-8d8d-4913-b30a-ec19f24998ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:23.468392 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-bf0c5371-8d8d-4913-b30a-ec19f24998ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:23.506190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a57e1377-3443-4296-a0d7-4484a4083053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'beee0e3d927642589c064aac5e697ff4', 'request_id': 'req-a57e1377-3443-4296-a0d7-4484a4083053'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:23.512731 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a57e1377-3443-4296-a0d7-4484a4083053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'beee0e3d927642589c064aac5e697ff4', 'request_id': 'req-a57e1377-3443-4296-a0d7-4484a4083053'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:23.519170 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a57e1377-3443-4296-a0d7-4484a4083053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'beee0e3d927642589c064aac5e697ff4', 'request_id': 'req-a57e1377-3443-4296-a0d7-4484a4083053'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:23.527677 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0c5371-8d8d-4913-b30a-ec19f24998ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:23.529114 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0c5371-8d8d-4913-b30a-ec19f24998ec service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:23.539205 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 985/1954] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:23 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:02:23.622753 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a57e1377-3443-4296-a0d7-4484a4083053 admin admin] The token cache is being invalidate because project beee0e3d927642589c064aac5e697ff4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:23.622983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a57e1377-3443-4296-a0d7-4484a4083053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:23.623714 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a57e1377-3443-4296-a0d7-4484a4083053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:23.625560 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 970/1955] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:02:23 2022] DELETE /identity/v3/projects/beee0e3d927642589c064aac5e697ff4 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:23.631209 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5bde8-3f24-413f-ac01-37c625e5f7dc None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:23.632001 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5bde8-3f24-413f-ac01-37c625e5f7dc None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:23.632506 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5bde8-3f24-413f-ac01-37c625e5f7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:23.653551 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-f6b5bde8-3f24-413f-ac01-37c625e5f7dc None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:23.655159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f6b5bde8-3f24-413f-ac01-37c625e5f7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:23.723865 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 986/1956] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:02:23.731444 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52c1e2-2b4c-44fa-a33b-106aace3ded3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:23.731746 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52c1e2-2b4c-44fa-a33b-106aace3ded3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:23.732027 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52c1e2-2b4c-44fa-a33b-106aace3ded3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:23.754694 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-2f52c1e2-2b4c-44fa-a33b-106aace3ded3 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:23.756047 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2f52c1e2-2b4c-44fa-a33b-106aace3ded3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:23.763935 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 971/1957] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:23.768932 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b5bde8-3f24-413f-ac01-37c625e5f7dc None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:23.778582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b5bde8-3f24-413f-ac01-37c625e5f7dc None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:23.784756 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89b1b2-2ded-45b0-bc6b-a29d6ba07c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:23.786606 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89b1b2-2ded-45b0-bc6b-a29d6ba07c6e admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:23.786965 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89b1b2-2ded-45b0-bc6b-a29d6ba07c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:23.789072 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89b1b2-2ded-45b0-bc6b-a29d6ba07c6e admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:23.789072 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89b1b2-2ded-45b0-bc6b-a29d6ba07c6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:02:23.789072 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89b1b2-2ded-45b0-bc6b-a29d6ba07c6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:23.790404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89b1b2-2ded-45b0-bc6b-a29d6ba07c6e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:23.804898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 987/1958] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:02:23.809190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52c1e2-2b4c-44fa-a33b-106aace3ded3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:23.821675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52c1e2-2b4c-44fa-a33b-106aace3ded3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:23.828892 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04817c-5fbe-41db-8d3e-1e131b0cb5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:23.830503 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04817c-5fbe-41db-8d3e-1e131b0cb5d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:23.831080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04817c-5fbe-41db-8d3e-1e131b0cb5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:23.831368 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04817c-5fbe-41db-8d3e-1e131b0cb5d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:23.832375 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04817c-5fbe-41db-8d3e-1e131b0cb5d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:23.835603 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04817c-5fbe-41db-8d3e-1e131b0cb5d1 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:23.945095 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 972/1959] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:02:23 2022] POST /identity/v3/projects => generated 381 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:23.949469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89b1b2-2ded-45b0-bc6b-a29d6ba07c6e admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:23.960317 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89b1b2-2ded-45b0-bc6b-a29d6ba07c6e admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:24.018027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c51489-6fbd-41d9-bf3f-22642b4e03f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:24.019263 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c51489-6fbd-41d9-bf3f-22642b4e03f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:24.019263 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c51489-6fbd-41d9-bf3f-22642b4e03f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:24.019263 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c51489-6fbd-41d9-bf3f-22642b4e03f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:24.019818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c51489-6fbd-41d9-bf3f-22642b4e03f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:24.021847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c51489-6fbd-41d9-bf3f-22642b4e03f2 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:24.126987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 988/1960] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:02:23 2022] POST /identity/v3/users => generated 291 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:02:24.129896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04817c-5fbe-41db-8d3e-1e131b0cb5d1 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:24.143719 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04817c-5fbe-41db-8d3e-1e131b0cb5d1 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:24.146252 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c51489-6fbd-41d9-bf3f-22642b4e03f2 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:24.148049 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-18c51489-6fbd-41d9-bf3f-22642b4e03f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:24.149448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec8881-9e0a-4847-a7fa-fe2d932d4c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:24.150035 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec8881-9e0a-4847-a7fa-fe2d932d4c54 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:24.151598 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec8881-9e0a-4847-a7fa-fe2d932d4c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:24.151598 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec8881-9e0a-4847-a7fa-fe2d932d4c54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:24.151598 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec8881-9e0a-4847-a7fa-fe2d932d4c54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:24.153018 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec8881-9e0a-4847-a7fa-fe2d932d4c54 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:24.160643 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c51489-6fbd-41d9-bf3f-22642b4e03f2 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:24.164118 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 973/1961] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:24.165588 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3877047-6bd2-49bb-b48f-4d42a87bd861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:24.166230 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3877047-6bd2-49bb-b48f-4d42a87bd861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:24.166481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3877047-6bd2-49bb-b48f-4d42a87bd861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:24.166786 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3877047-6bd2-49bb-b48f-4d42a87bd861 admin admin] PATH_INFO: `/v3/users/82e761c620704517a292482256589d08` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:24.167912 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3877047-6bd2-49bb-b48f-4d42a87bd861 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82e761c620704517a292482256589d08)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:24.168926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec8881-9e0a-4847-a7fa-fe2d932d4c54 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:24.170034 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3877047-6bd2-49bb-b48f-4d42a87bd861 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:24.179473 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec8881-9e0a-4847-a7fa-fe2d932d4c54 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:24.184095 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862231c-b114-456c-8682-2234b07ecad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:24.184763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862231c-b114-456c-8682-2234b07ecad7 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:24.185039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862231c-b114-456c-8682-2234b07ecad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:24.185305 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862231c-b114-456c-8682-2234b07ecad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:24.185936 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f862231c-b114-456c-8682-2234b07ecad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:24.187630 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f862231c-b114-456c-8682-2234b07ecad7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:24.198235 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 974/1962] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:24.202507 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862231c-b114-456c-8682-2234b07ecad7 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:24.212955 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862231c-b114-456c-8682-2234b07ecad7 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:24.217546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be138e-fa7a-467d-b535-b1d20440379e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:24.218177 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be138e-fa7a-467d-b535-b1d20440379e admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:24.218453 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be138e-fa7a-467d-b535-b1d20440379e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:24.218759 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be138e-fa7a-467d-b535-b1d20440379e admin admin] PATH_INFO: `/v3/projects/d2029a9d5a294561b55de8901348554e/users/7eade3fb48284b088a1b2ce5e74e9383/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:24.234087 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be138e-fa7a-467d-b535-b1d20440379e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2029a9d5a294561b55de8901348554e, user_id=7eade3fb48284b088a1b2ce5e74e9383, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:24.237586 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be138e-fa7a-467d-b535-b1d20440379e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:24.326253 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 975/1963] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:24 2022] PUT /identity/v3/projects/d2029a9d5a294561b55de8901348554e/users/7eade3fb48284b088a1b2ce5e74e9383/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:02:24.337657 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40844339-3999-4d6a-9d8c-cc0481fc25c2 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:24.337941 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40844339-3999-4d6a-9d8c-cc0481fc25c2 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:24.338213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40844339-3999-4d6a-9d8c-cc0481fc25c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:24.361871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e3877047-6bd2-49bb-b48f-4d42a87bd861 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82e761c620704517a292482256589d08'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:24.368628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-40844339-3999-4d6a-9d8c-cc0481fc25c2 None None] MFA Rules not processed for user `7eade3fb48284b088a1b2ce5e74e9383`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:24.369855 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-40844339-3999-4d6a-9d8c-cc0481fc25c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:24.371510 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:24.376774 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e3877047-6bd2-49bb-b48f-4d42a87bd861 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82e761c620704517a292482256589d08'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:24.387332 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e3877047-6bd2-49bb-b48f-4d42a87bd861 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82e761c620704517a292482256589d08'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:24.388211 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e3877047-6bd2-49bb-b48f-4d42a87bd861 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82e761c620704517a292482256589d08'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:24.455636 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 976/1964] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:02:24.459870 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40844339-3999-4d6a-9d8c-cc0481fc25c2 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:24.461213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-40844339-3999-4d6a-9d8c-cc0481fc25c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:24.476139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:24.486218 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e3877047-6bd2-49bb-b48f-4d42a87bd861 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82e761c620704517a292482256589d08'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:24.493098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 989/1965] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:02:24 2022] DELETE /identity/v3/users/82e761c620704517a292482256589d08 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:24.497472 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423c156-23a2-49d0-ade6-4fb08c933d2e None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:24.497758 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423c156-23a2-49d0-ade6-4fb08c933d2e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:24.498097 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423c156-23a2-49d0-ade6-4fb08c933d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:24.524522 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-e423c156-23a2-49d0-ade6-4fb08c933d2e None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:24.525841 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e423c156-23a2-49d0-ade6-4fb08c933d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:24.533979 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40844339-3999-4d6a-9d8c-cc0481fc25c2 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:24.594934 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e897dcc-b3a9-4e20-9ee6-b40967679df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:24.596076 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e897dcc-b3a9-4e20-9ee6-b40967679df1 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:24.596363 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e897dcc-b3a9-4e20-9ee6-b40967679df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:24.596786 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e897dcc-b3a9-4e20-9ee6-b40967679df1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:24.597258 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e897dcc-b3a9-4e20-9ee6-b40967679df1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:24.599275 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e897dcc-b3a9-4e20-9ee6-b40967679df1 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:24.603706 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 990/1966] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:02:24.620603 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423c156-23a2-49d0-ade6-4fb08c933d2e None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:24.622017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e423c156-23a2-49d0-ade6-4fb08c933d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:24.681209 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423c156-23a2-49d0-ade6-4fb08c933d2e None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:24.684927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609224e-5e68-4ce1-82df-7e82b78df516 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:24.685550 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609224e-5e68-4ce1-82df-7e82b78df516 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:24.685818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609224e-5e68-4ce1-82df-7e82b78df516 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:24.686089 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609224e-5e68-4ce1-82df-7e82b78df516 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:24.695797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d609224e-5e68-4ce1-82df-7e82b78df516 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:24.697219 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d609224e-5e68-4ce1-82df-7e82b78df516 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:24.706996 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 991/1967] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:02:24.713920 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 977/1968] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:02:24 2022] POST /identity/v3/projects => generated 379 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:02:24.718759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609224e-5e68-4ce1-82df-7e82b78df516 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:24.730312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609224e-5e68-4ce1-82df-7e82b78df516 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:24.741561 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:24.793613 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c00bab-54e1-41a3-ac73-356b8416849f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:24.794231 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c00bab-54e1-41a3-ac73-356b8416849f admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:24.794510 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c00bab-54e1-41a3-ac73-356b8416849f admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:24.795045 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c00bab-54e1-41a3-ac73-356b8416849f admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:24.795921 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c00bab-54e1-41a3-ac73-356b8416849f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:24.797962 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c00bab-54e1-41a3-ac73-356b8416849f admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:24.909285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 992/1969] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:02:24 2022] POST /identity/v3/users => generated 296 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:24.913230 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e897dcc-b3a9-4e20-9ee6-b40967679df1 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:24.935120 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e897dcc-b3a9-4e20-9ee6-b40967679df1 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:24.935904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6dd18e-ce95-4c00-8a57-908e159dbd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:24.936528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6dd18e-ce95-4c00-8a57-908e159dbd04 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:24.936802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6dd18e-ce95-4c00-8a57-908e159dbd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:24.937090 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6dd18e-ce95-4c00-8a57-908e159dbd04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:24.937748 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6dd18e-ce95-4c00-8a57-908e159dbd04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:24.940286 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6dd18e-ce95-4c00-8a57-908e159dbd04 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:24.951710 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 978/1970] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:24.955971 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c00bab-54e1-41a3-ac73-356b8416849f admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:24.965906 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c00bab-54e1-41a3-ac73-356b8416849f admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:24.970620 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc26664-8572-4fb8-b0e1-3ddcdde61e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:24.971381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc26664-8572-4fb8-b0e1-3ddcdde61e58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:24.971665 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc26664-8572-4fb8-b0e1-3ddcdde61e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:24.971931 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc26664-8572-4fb8-b0e1-3ddcdde61e58 admin admin] PATH_INFO: `/v3/projects/31556c31a81a4895bed0e6493b32bc71/users/5a444a2c54d147bc857efc59c2bb29bb/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:24.988092 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc26664-8572-4fb8-b0e1-3ddcdde61e58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31556c31a81a4895bed0e6493b32bc71, user_id=5a444a2c54d147bc857efc59c2bb29bb, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:24.990850 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc26664-8572-4fb8-b0e1-3ddcdde61e58 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:25.075511 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 993/1971] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:24 2022] PUT /identity/v3/projects/31556c31a81a4895bed0e6493b32bc71/users/5a444a2c54d147bc857efc59c2bb29bb/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:25.079460 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6dd18e-ce95-4c00-8a57-908e159dbd04 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:25.090289 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6dd18e-ce95-4c00-8a57-908e159dbd04 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:25.153440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795059dc-2769-40c6-9df2-2ae2474e40ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:25.153960 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795059dc-2769-40c6-9df2-2ae2474e40ec admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:25.154238 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795059dc-2769-40c6-9df2-2ae2474e40ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:25.154507 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795059dc-2769-40c6-9df2-2ae2474e40ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:25.155172 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795059dc-2769-40c6-9df2-2ae2474e40ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:25.156776 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795059dc-2769-40c6-9df2-2ae2474e40ec admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:25.167106 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 979/1972] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:25.171485 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc26664-8572-4fb8-b0e1-3ddcdde61e58 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:25.187518 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc26664-8572-4fb8-b0e1-3ddcdde61e58 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:25.195510 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39b5a8-1659-4472-8620-cbdc15a507d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:25.196291 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39b5a8-1659-4472-8620-cbdc15a507d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:25.196949 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39b5a8-1659-4472-8620-cbdc15a507d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:25.197375 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39b5a8-1659-4472-8620-cbdc15a507d8 admin admin] PATH_INFO: `/v3/projects/31556c31a81a4895bed0e6493b32bc71/users/5a444a2c54d147bc857efc59c2bb29bb/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:25.199796 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e39b5a8-1659-4472-8620-cbdc15a507d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31556c31a81a4895bed0e6493b32bc71, user_id=5a444a2c54d147bc857efc59c2bb29bb, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:25.202701 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e39b5a8-1659-4472-8620-cbdc15a507d8 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:25.217577 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 994/1973] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:25 2022] PUT /identity/v3/projects/31556c31a81a4895bed0e6493b32bc71/users/5a444a2c54d147bc857efc59c2bb29bb/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:25.228280 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4a262-4e04-4cac-802a-2fdf07d5015a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:25.233684 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4a262-4e04-4cac-802a-2fdf07d5015a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:25.233684 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4a262-4e04-4cac-802a-2fdf07d5015a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:25.272094 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5ee4a262-4e04-4cac-802a-2fdf07d5015a None None] MFA Rules not processed for user `5a444a2c54d147bc857efc59c2bb29bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:25.273841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5ee4a262-4e04-4cac-802a-2fdf07d5015a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:25.323616 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:25.370137 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 980/1974] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:25 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:02:25.381901 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39b5a8-1659-4472-8620-cbdc15a507d8 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:25.383791 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0e39b5a8-1659-4472-8620-cbdc15a507d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:25.444547 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4a262-4e04-4cac-802a-2fdf07d5015a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:25.445994 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5ee4a262-4e04-4cac-802a-2fdf07d5015a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:25.454636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39b5a8-1659-4472-8620-cbdc15a507d8 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:25.460282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68066f5-ebe0-43de-8fb0-3cdd6fc86f31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:25.461161 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68066f5-ebe0-43de-8fb0-3cdd6fc86f31 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:25.461553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68066f5-ebe0-43de-8fb0-3cdd6fc86f31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:25.461825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68066f5-ebe0-43de-8fb0-3cdd6fc86f31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:25.475210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68066f5-ebe0-43de-8fb0-3cdd6fc86f31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:25.477938 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68066f5-ebe0-43de-8fb0-3cdd6fc86f31 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:25.493183 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 995/1975] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:25 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:02:25.521908 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4a262-4e04-4cac-802a-2fdf07d5015a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:25.527406 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7014db7-82e9-405a-919a-e964fce73836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:25.528232 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7014db7-82e9-405a-919a-e964fce73836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:25.528577 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7014db7-82e9-405a-919a-e964fce73836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:25.528770 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7014db7-82e9-405a-919a-e964fce73836 admin admin] PATH_INFO: `/v3/projects/5f2005ed586648d1b13872c45324b5f0` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:25.529898 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7014db7-82e9-405a-919a-e964fce73836 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f2005ed586648d1b13872c45324b5f0)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:25.532038 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7014db7-82e9-405a-919a-e964fce73836 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:25.628514 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b7014db7-82e9-405a-919a-e964fce73836 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f2005ed586648d1b13872c45324b5f0', 'request_id': 'req-b7014db7-82e9-405a-919a-e964fce73836'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:25.636043 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b7014db7-82e9-405a-919a-e964fce73836 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f2005ed586648d1b13872c45324b5f0', 'request_id': 'req-b7014db7-82e9-405a-919a-e964fce73836'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:25.641703 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b7014db7-82e9-405a-919a-e964fce73836 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f2005ed586648d1b13872c45324b5f0', 'request_id': 'req-b7014db7-82e9-405a-919a-e964fce73836'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:25.713179 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68066f5-ebe0-43de-8fb0-3cdd6fc86f31 service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:25.714581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a68066f5-ebe0-43de-8fb0-3cdd6fc86f31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:25.727572 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68066f5-ebe0-43de-8fb0-3cdd6fc86f31 service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:25.732953 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b7014db7-82e9-405a-919a-e964fce73836 admin admin] The token cache is being invalidate because project 5f2005ed586648d1b13872c45324b5f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:25.733252 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b7014db7-82e9-405a-919a-e964fce73836 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:25.733965 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b7014db7-82e9-405a-919a-e964fce73836 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:25.737007 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 981/1976] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:02:25 2022] DELETE /identity/v3/projects/5f2005ed586648d1b13872c45324b5f0 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:25.740892 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7014db7-82e9-405a-919a-e964fce73836 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:25.742370 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b7014db7-82e9-405a-919a-e964fce73836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:25.794985 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d00784-653c-4882-be1c-e6d898498923 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:25.795611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d00784-653c-4882-be1c-e6d898498923 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:25.795881 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d00784-653c-4882-be1c-e6d898498923 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:25.796147 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d00784-653c-4882-be1c-e6d898498923 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:25.807522 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-68d00784-653c-4882-be1c-e6d898498923 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:25.825071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:25.844235 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:25.847628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7014db7-82e9-405a-919a-e964fce73836 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:25.864492 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d55c2-e995-4a0a-b1ba-792a7b296841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:25.865220 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d55c2-e995-4a0a-b1ba-792a7b296841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:25.865524 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d55c2-e995-4a0a-b1ba-792a7b296841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:25.865790 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d55c2-e995-4a0a-b1ba-792a7b296841 admin admin] PATH_INFO: `/v3/users/7d02c6d1d7f440f18a692e53de82947a` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:25.867228 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d55c2-e995-4a0a-b1ba-792a7b296841 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d02c6d1d7f440f18a692e53de82947a)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:25.869489 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d55c2-e995-4a0a-b1ba-792a7b296841 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:25.911189 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d00784-653c-4882-be1c-e6d898498923 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:25.913097 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d00784-653c-4882-be1c-e6d898498923 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:25.926253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 996/1977] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:02:26.130505 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-290d55c2-e995-4a0a-b1ba-792a7b296841 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d02c6d1d7f440f18a692e53de82947a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:26.137294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-290d55c2-e995-4a0a-b1ba-792a7b296841 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d02c6d1d7f440f18a692e53de82947a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:26.150822 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-290d55c2-e995-4a0a-b1ba-792a7b296841 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d02c6d1d7f440f18a692e53de82947a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:26.239577 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-290d55c2-e995-4a0a-b1ba-792a7b296841 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d02c6d1d7f440f18a692e53de82947a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:26.240303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-290d55c2-e995-4a0a-b1ba-792a7b296841 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d02c6d1d7f440f18a692e53de82947a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:26.247703 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 982/1978] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:02:25 2022] DELETE /identity/v3/users/7d02c6d1d7f440f18a692e53de82947a => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:26.737722 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d00784-653c-4882-be1c-e6d898498923 service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:26.756900 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d00784-653c-4882-be1c-e6d898498923 service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:26.818611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec654da7-124a-4ae7-a36e-c4066a75504b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:26.819505 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec654da7-124a-4ae7-a36e-c4066a75504b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:26.819505 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec654da7-124a-4ae7-a36e-c4066a75504b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:26.819791 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec654da7-124a-4ae7-a36e-c4066a75504b admin admin] PATH_INFO: `/v3/projects/7f964f2ae98e4e5aaf778cdcaddae72a` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:26.820991 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec654da7-124a-4ae7-a36e-c4066a75504b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f964f2ae98e4e5aaf778cdcaddae72a)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:26.822895 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec654da7-124a-4ae7-a36e-c4066a75504b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:26.967941 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ec654da7-124a-4ae7-a36e-c4066a75504b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f964f2ae98e4e5aaf778cdcaddae72a', 'request_id': 'req-ec654da7-124a-4ae7-a36e-c4066a75504b'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:27.061445 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ec654da7-124a-4ae7-a36e-c4066a75504b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f964f2ae98e4e5aaf778cdcaddae72a', 'request_id': 'req-ec654da7-124a-4ae7-a36e-c4066a75504b'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:27.068335 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ec654da7-124a-4ae7-a36e-c4066a75504b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f964f2ae98e4e5aaf778cdcaddae72a', 'request_id': 'req-ec654da7-124a-4ae7-a36e-c4066a75504b'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:27.079980 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ec654da7-124a-4ae7-a36e-c4066a75504b admin admin] The token cache is being invalidate because project 7f964f2ae98e4e5aaf778cdcaddae72a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:27.080447 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ec654da7-124a-4ae7-a36e-c4066a75504b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:27.081208 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ec654da7-124a-4ae7-a36e-c4066a75504b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:27.082734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 997/1979] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:02:26 2022] DELETE /identity/v3/projects/7f964f2ae98e4e5aaf778cdcaddae72a => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:27.087546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa0662-a631-4522-9062-c90041bebcd5 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:27.087997 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa0662-a631-4522-9062-c90041bebcd5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:27.088101 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa0662-a631-4522-9062-c90041bebcd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:27.108673 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-35fa0662-a631-4522-9062-c90041bebcd5 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:27.109882 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-35fa0662-a631-4522-9062-c90041bebcd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:27.180603 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 983/1980] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:27.188763 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58077c-05e8-460f-99f5-208109f82bcf None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:27.189043 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58077c-05e8-460f-99f5-208109f82bcf None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:27.189394 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58077c-05e8-460f-99f5-208109f82bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:27.211057 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-1a58077c-05e8-460f-99f5-208109f82bcf None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:27.212265 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1a58077c-05e8-460f-99f5-208109f82bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:27.221015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 998/1981] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:27.225889 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa0662-a631-4522-9062-c90041bebcd5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:27.236402 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa0662-a631-4522-9062-c90041bebcd5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:27.241503 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aedd19b-c973-4399-bdad-5728028423a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:27.242084 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aedd19b-c973-4399-bdad-5728028423a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:27.244153 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aedd19b-c973-4399-bdad-5728028423a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:27.244153 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aedd19b-c973-4399-bdad-5728028423a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:27.244153 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aedd19b-c973-4399-bdad-5728028423a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:02:27.244153 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aedd19b-c973-4399-bdad-5728028423a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:27.245397 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aedd19b-c973-4399-bdad-5728028423a4 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:27.261084 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 984/1982] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:02:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:02:27.262432 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58077c-05e8-460f-99f5-208109f82bcf None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:27.275066 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58077c-05e8-460f-99f5-208109f82bcf None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:27.282795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2744c4e1-9b9d-4c55-bb3d-6d484e424c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:27.284430 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744c4e1-9b9d-4c55-bb3d-6d484e424c81 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:27.284698 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744c4e1-9b9d-4c55-bb3d-6d484e424c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:27.284966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744c4e1-9b9d-4c55-bb3d-6d484e424c81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:27.286172 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2744c4e1-9b9d-4c55-bb3d-6d484e424c81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:27.288124 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2744c4e1-9b9d-4c55-bb3d-6d484e424c81 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:27.438823 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 999/1983] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:02:27 2022] POST /identity/v3/projects => generated 377 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:27.440026 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aedd19b-c973-4399-bdad-5728028423a4 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:27.450424 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aedd19b-c973-4399-bdad-5728028423a4 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:27.499933 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:27.513433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799c79b-76dd-47d5-a420-3f1d335d199c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:27.513433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799c79b-76dd-47d5-a420-3f1d335d199c admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:27.513433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799c79b-76dd-47d5-a420-3f1d335d199c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:27.513433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799c79b-76dd-47d5-a420-3f1d335d199c admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:27.513817 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f799c79b-76dd-47d5-a420-3f1d335d199c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:27.513817 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f799c79b-76dd-47d5-a420-3f1d335d199c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:27.593764 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 985/1984] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:02:27 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:27.598011 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2744c4e1-9b9d-4c55-bb3d-6d484e424c81 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:27.608330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2744c4e1-9b9d-4c55-bb3d-6d484e424c81 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:27.614641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3891b4b-c5a9-4238-bf8c-ffe6e9665a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:27.615286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3891b4b-c5a9-4238-bf8c-ffe6e9665a22 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:27.615562 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3891b4b-c5a9-4238-bf8c-ffe6e9665a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:27.615823 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3891b4b-c5a9-4238-bf8c-ffe6e9665a22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:27.616504 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3891b4b-c5a9-4238-bf8c-ffe6e9665a22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:27.618044 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3891b4b-c5a9-4238-bf8c-ffe6e9665a22 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:27.630722 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1000/1985] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:27.635910 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799c79b-76dd-47d5-a420-3f1d335d199c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:27.647019 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799c79b-76dd-47d5-a420-3f1d335d199c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:27.651768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6114ee-fc1c-45f0-a36d-b7b59aed9335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:27.652365 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6114ee-fc1c-45f0-a36d-b7b59aed9335 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:27.652653 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6114ee-fc1c-45f0-a36d-b7b59aed9335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:27.652913 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6114ee-fc1c-45f0-a36d-b7b59aed9335 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:27.653644 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6114ee-fc1c-45f0-a36d-b7b59aed9335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:27.655307 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6114ee-fc1c-45f0-a36d-b7b59aed9335 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:27.670539 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3891b4b-c5a9-4238-bf8c-ffe6e9665a22 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:27.671974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 986/1986] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:27.681624 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3891b4b-c5a9-4238-bf8c-ffe6e9665a22 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:27.687847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367927cf-1ccf-4937-bfed-cbbff6e1bfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:27.688829 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367927cf-1ccf-4937-bfed-cbbff6e1bfe9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:27.690283 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367927cf-1ccf-4937-bfed-cbbff6e1bfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:27.691247 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367927cf-1ccf-4937-bfed-cbbff6e1bfe9 admin admin] PATH_INFO: `/v3/projects/d14dbcdf01214d10bb717a3e9727b7d6/users/057f4d33e4724c4995c27dbd5a2a9026/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:27.705128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:27.709240 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367927cf-1ccf-4937-bfed-cbbff6e1bfe9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d14dbcdf01214d10bb717a3e9727b7d6, user_id=057f4d33e4724c4995c27dbd5a2a9026, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:27.712396 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367927cf-1ccf-4937-bfed-cbbff6e1bfe9 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:27.784096 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1001/1987] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:27 2022] PUT /identity/v3/projects/d14dbcdf01214d10bb717a3e9727b7d6/users/057f4d33e4724c4995c27dbd5a2a9026/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:27.784515 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631bd2a7-0975-4a56-aa54-e0a94b83084d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:27.784947 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631bd2a7-0975-4a56-aa54-e0a94b83084d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:27.785334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631bd2a7-0975-4a56-aa54-e0a94b83084d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:27.824924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-631bd2a7-0975-4a56-aa54-e0a94b83084d None None] MFA Rules not processed for user `057f4d33e4724c4995c27dbd5a2a9026`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:27.826515 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-631bd2a7-0975-4a56-aa54-e0a94b83084d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:27.897265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 987/1988] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:27.910040 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367927cf-1ccf-4937-bfed-cbbff6e1bfe9 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:27.912015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-367927cf-1ccf-4937-bfed-cbbff6e1bfe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:27.973810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367927cf-1ccf-4937-bfed-cbbff6e1bfe9 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:27.978068 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104785c8-35e4-41bb-8cdc-3e1628b05e8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:27.978684 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104785c8-35e4-41bb-8cdc-3e1628b05e8d service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:27.978974 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104785c8-35e4-41bb-8cdc-3e1628b05e8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:27.979245 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104785c8-35e4-41bb-8cdc-3e1628b05e8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:27.988890 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104785c8-35e4-41bb-8cdc-3e1628b05e8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:27.990334 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104785c8-35e4-41bb-8cdc-3e1628b05e8d service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:27.999274 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1002/1989] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:02:30.545566 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631bd2a7-0975-4a56-aa54-e0a94b83084d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:30.546994 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-631bd2a7-0975-4a56-aa54-e0a94b83084d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:30.559660 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631bd2a7-0975-4a56-aa54-e0a94b83084d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:30.563209 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80449c7-b6ce-4e7b-bf6c-69c1d4d2bb04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:30.563797 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80449c7-b6ce-4e7b-bf6c-69c1d4d2bb04 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:30.564075 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80449c7-b6ce-4e7b-bf6c-69c1d4d2bb04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:30.564344 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80449c7-b6ce-4e7b-bf6c-69c1d4d2bb04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:30.566534 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d80449c7-b6ce-4e7b-bf6c-69c1d4d2bb04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:30.626587 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80449c7-b6ce-4e7b-bf6c-69c1d4d2bb04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:30.628452 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80449c7-b6ce-4e7b-bf6c-69c1d4d2bb04 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:30.637489 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 988/1990] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:30 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:02:32.345210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3111447-d137-4f3f-8415-ebacd7876325 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:32.345811 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3111447-d137-4f3f-8415-ebacd7876325 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:32.345922 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3111447-d137-4f3f-8415-ebacd7876325 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:32.347681 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1003/1991] 10.222.0.17 () {54 vars in 921 bytes} [Sun Sep 11 00:02:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:02:32.389160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55175301-daae-4b45-ba6c-6799486b0102 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:32.389616 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55175301-daae-4b45-ba6c-6799486b0102 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:32.389693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55175301-daae-4b45-ba6c-6799486b0102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:32.410710 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-55175301-daae-4b45-ba6c-6799486b0102 None None] MFA Rules not processed for user `5cb1d85a338e40f18e61887a1becb906`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:32.411996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-55175301-daae-4b45-ba6c-6799486b0102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:32.480564 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:32.487724 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 989/1992] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Sep 11 00:02:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:32.498714 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3111447-d137-4f3f-8415-ebacd7876325 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:32.500477 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e3111447-d137-4f3f-8415-ebacd7876325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:32.516478 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3111447-d137-4f3f-8415-ebacd7876325 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:32.523409 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7972f043-1b53-44d8-8e9c-5b00cc745cda service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cb1d85a338e40f18e61887a1becb906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:32.524021 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972f043-1b53-44d8-8e9c-5b00cc745cda service placement] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:32.524362 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972f043-1b53-44d8-8e9c-5b00cc745cda service placement] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:32.524699 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972f043-1b53-44d8-8e9c-5b00cc745cda service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:32.534738 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7972f043-1b53-44d8-8e9c-5b00cc745cda service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:32.536688 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7972f043-1b53-44d8-8e9c-5b00cc745cda service placement] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:32.559786 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1004/1993] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:02:33.576445 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55175301-daae-4b45-ba6c-6799486b0102 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:33.581214 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-55175301-daae-4b45-ba6c-6799486b0102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:33.671938 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55175301-daae-4b45-ba6c-6799486b0102 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:33.678103 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c3928-b0fa-4b72-b05c-e4eed649ffce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:33.679036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c3928-b0fa-4b72-b05c-e4eed649ffce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:33.679491 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c3928-b0fa-4b72-b05c-e4eed649ffce admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:33.679934 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c3928-b0fa-4b72-b05c-e4eed649ffce admin admin] PATH_INFO: `/v3/users/a507ed547e534d5e9a0f15d5d7ce9dbd` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:33.681538 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c3928-b0fa-4b72-b05c-e4eed649ffce admin admin] RBAC: Authorizing `identity:delete_user(user_id=a507ed547e534d5e9a0f15d5d7ce9dbd)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:33.683714 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c3928-b0fa-4b72-b05c-e4eed649ffce admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:33.899223 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-228c3928-b0fa-4b72-b05c-e4eed649ffce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a507ed547e534d5e9a0f15d5d7ce9dbd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:33.918163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-228c3928-b0fa-4b72-b05c-e4eed649ffce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a507ed547e534d5e9a0f15d5d7ce9dbd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:33.934348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-228c3928-b0fa-4b72-b05c-e4eed649ffce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a507ed547e534d5e9a0f15d5d7ce9dbd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:34.021005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-228c3928-b0fa-4b72-b05c-e4eed649ffce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a507ed547e534d5e9a0f15d5d7ce9dbd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:34.021741 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-228c3928-b0fa-4b72-b05c-e4eed649ffce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a507ed547e534d5e9a0f15d5d7ce9dbd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:34.028842 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 990/1994] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:02:33 2022] DELETE /identity/v3/users/a507ed547e534d5e9a0f15d5d7ce9dbd => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:02:34.275546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7972f043-1b53-44d8-8e9c-5b00cc745cda service placement] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:34.288326 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7972f043-1b53-44d8-8e9c-5b00cc745cda service placement] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:34.375610 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835849ef-1f15-465e-8fee-4f67ecf1fd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:34.376415 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835849ef-1f15-465e-8fee-4f67ecf1fd13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:34.377357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835849ef-1f15-465e-8fee-4f67ecf1fd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:34.377728 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835849ef-1f15-465e-8fee-4f67ecf1fd13 admin admin] PATH_INFO: `/v3/projects/15b185408d9440d5a1b60ddb0bd35809` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:34.379259 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835849ef-1f15-465e-8fee-4f67ecf1fd13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15b185408d9440d5a1b60ddb0bd35809)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:34.382419 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835849ef-1f15-465e-8fee-4f67ecf1fd13 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:34.538163 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-835849ef-1f15-465e-8fee-4f67ecf1fd13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15b185408d9440d5a1b60ddb0bd35809', 'request_id': 'req-835849ef-1f15-465e-8fee-4f67ecf1fd13'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:34.615710 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-835849ef-1f15-465e-8fee-4f67ecf1fd13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15b185408d9440d5a1b60ddb0bd35809', 'request_id': 'req-835849ef-1f15-465e-8fee-4f67ecf1fd13'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:34.622835 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-835849ef-1f15-465e-8fee-4f67ecf1fd13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15b185408d9440d5a1b60ddb0bd35809', 'request_id': 'req-835849ef-1f15-465e-8fee-4f67ecf1fd13'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:34.652805 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-835849ef-1f15-465e-8fee-4f67ecf1fd13 admin admin] The token cache is being invalidate because project 15b185408d9440d5a1b60ddb0bd35809 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:34.653335 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-835849ef-1f15-465e-8fee-4f67ecf1fd13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:34.654498 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-835849ef-1f15-465e-8fee-4f67ecf1fd13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:34.656751 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1005/1995] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:02:34 2022] DELETE /identity/v3/projects/15b185408d9440d5a1b60ddb0bd35809 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:34.660998 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c3928-b0fa-4b72-b05c-e4eed649ffce admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:34.663224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-228c3928-b0fa-4b72-b05c-e4eed649ffce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:34.680780 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c3928-b0fa-4b72-b05c-e4eed649ffce admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:34.773571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce319049-7953-45c9-8a6c-7a9f1f01f9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:34.774214 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce319049-7953-45c9-8a6c-7a9f1f01f9e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:34.774487 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce319049-7953-45c9-8a6c-7a9f1f01f9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:34.774789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce319049-7953-45c9-8a6c-7a9f1f01f9e7 admin admin] PATH_INFO: `/v3/users/e24e5ae5119848cb9b430ba243037c74` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:34.776236 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce319049-7953-45c9-8a6c-7a9f1f01f9e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e24e5ae5119848cb9b430ba243037c74)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:34.778991 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce319049-7953-45c9-8a6c-7a9f1f01f9e7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:34.796931 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:35.049226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ce319049-7953-45c9-8a6c-7a9f1f01f9e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e24e5ae5119848cb9b430ba243037c74'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:35.057859 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ce319049-7953-45c9-8a6c-7a9f1f01f9e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e24e5ae5119848cb9b430ba243037c74'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:35.073304 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ce319049-7953-45c9-8a6c-7a9f1f01f9e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e24e5ae5119848cb9b430ba243037c74'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:35.153839 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ce319049-7953-45c9-8a6c-7a9f1f01f9e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e24e5ae5119848cb9b430ba243037c74'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:35.154643 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ce319049-7953-45c9-8a6c-7a9f1f01f9e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e24e5ae5119848cb9b430ba243037c74'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:35.167564 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 991/1996] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:02:34 2022] DELETE /identity/v3/users/e24e5ae5119848cb9b430ba243037c74 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:35.471432 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835849ef-1f15-465e-8fee-4f67ecf1fd13 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:35.473399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-835849ef-1f15-465e-8fee-4f67ecf1fd13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:35.507186 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835849ef-1f15-465e-8fee-4f67ecf1fd13 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:35.514540 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:35.594383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f22fc-2d09-4be0-b7c5-d2cf2615fed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:35.597013 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f22fc-2d09-4be0-b7c5-d2cf2615fed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:35.606152 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f22fc-2d09-4be0-b7c5-d2cf2615fed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:35.606152 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f22fc-2d09-4be0-b7c5-d2cf2615fed1 admin admin] PATH_INFO: `/v3/projects/1ead4d3719db4002b3be221143d12994` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:35.607595 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2f22fc-2d09-4be0-b7c5-d2cf2615fed1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ead4d3719db4002b3be221143d12994)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:35.609429 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2f22fc-2d09-4be0-b7c5-d2cf2615fed1 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:35.744900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af2657f-b49e-4419-b92b-14cb36292e73 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:35.745442 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af2657f-b49e-4419-b92b-14cb36292e73 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:35.745539 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af2657f-b49e-4419-b92b-14cb36292e73 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:35.746968 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 992/1997] 10.222.0.17 () {54 vars in 921 bytes} [Sun Sep 11 00:02:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 00:02:35.749632 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-3b2f22fc-2d09-4be0-b7c5-d2cf2615fed1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ead4d3719db4002b3be221143d12994', 'request_id': 'req-3b2f22fc-2d09-4be0-b7c5-d2cf2615fed1'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:35.759615 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348adeb3-c4d7-42b0-806e-3371c5dd2471 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:35.759897 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348adeb3-c4d7-42b0-806e-3371c5dd2471 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:35.763862 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348adeb3-c4d7-42b0-806e-3371c5dd2471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:35.781008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-348adeb3-c4d7-42b0-806e-3371c5dd2471 None None] MFA Rules not processed for user `5cb1d85a338e40f18e61887a1becb906`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:35.782315 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-348adeb3-c4d7-42b0-806e-3371c5dd2471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:35.849865 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 993/1998] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Sep 11 00:02:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:02:35.856084 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-3b2f22fc-2d09-4be0-b7c5-d2cf2615fed1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ead4d3719db4002b3be221143d12994', 'request_id': 'req-3b2f22fc-2d09-4be0-b7c5-d2cf2615fed1'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:35.861600 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348adeb3-c4d7-42b0-806e-3371c5dd2471 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:35.863966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-348adeb3-c4d7-42b0-806e-3371c5dd2471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:35.865382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-3b2f22fc-2d09-4be0-b7c5-d2cf2615fed1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ead4d3719db4002b3be221143d12994', 'request_id': 'req-3b2f22fc-2d09-4be0-b7c5-d2cf2615fed1'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:35.877714 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-3b2f22fc-2d09-4be0-b7c5-d2cf2615fed1 admin admin] The token cache is being invalidate because project 1ead4d3719db4002b3be221143d12994 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:35.878337 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348adeb3-c4d7-42b0-806e-3371c5dd2471 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:35.884413 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-3b2f22fc-2d09-4be0-b7c5-d2cf2615fed1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:35.884413 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-3b2f22fc-2d09-4be0-b7c5-d2cf2615fed1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:35.884413 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1006/1999] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:02:35 2022] DELETE /identity/v3/projects/1ead4d3719db4002b3be221143d12994 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:35.888034 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa1423-4390-46f4-a2d2-458a7598d9c7 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:35.888687 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa1423-4390-46f4-a2d2-458a7598d9c7 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:35.889156 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa1423-4390-46f4-a2d2-458a7598d9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:35.892083 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477332b-0ced-49c3-a3c4-602babbb9346 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cb1d85a338e40f18e61887a1becb906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:35.893458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477332b-0ced-49c3-a3c4-602babbb9346 service placement] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:35.893893 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477332b-0ced-49c3-a3c4-602babbb9346 service placement] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:35.894795 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477332b-0ced-49c3-a3c4-602babbb9346 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:35.896792 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2477332b-0ced-49c3-a3c4-602babbb9346 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:35.911987 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2477332b-0ced-49c3-a3c4-602babbb9346 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:35.913944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2477332b-0ced-49c3-a3c4-602babbb9346 service placement] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:35.914749 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-abaa1423-4390-46f4-a2d2-458a7598d9c7 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:35.916332 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-abaa1423-4390-46f4-a2d2-458a7598d9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:35.929230 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 994/2000] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:02:35.930185 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477332b-0ced-49c3-a3c4-602babbb9346 service placement] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:35.931653 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2477332b-0ced-49c3-a3c4-602babbb9346 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:36.033542 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1007/2001] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:36.041499 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb404ea0-7cb4-409e-b54a-91f4e6482108 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:36.041899 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb404ea0-7cb4-409e-b54a-91f4e6482108 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:36.042160 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb404ea0-7cb4-409e-b54a-91f4e6482108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:36.076789 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-eb404ea0-7cb4-409e-b54a-91f4e6482108 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:36.078063 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-eb404ea0-7cb4-409e-b54a-91f4e6482108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:36.079526 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477332b-0ced-49c3-a3c4-602babbb9346 service placement] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:36.086585 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1008/2002] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:02:36.087497 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf367f1-d316-4315-82ec-ac4bb87fb0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:36.088348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf367f1-d316-4315-82ec-ac4bb87fb0b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:36.088625 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf367f1-d316-4315-82ec-ac4bb87fb0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:36.088927 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf367f1-d316-4315-82ec-ac4bb87fb0b0 admin admin] PATH_INFO: `/v3/users/640a5815b1a545fc952ac67f553b6797` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:36.090080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf367f1-d316-4315-82ec-ac4bb87fb0b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=640a5815b1a545fc952ac67f553b6797)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:36.091112 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb404ea0-7cb4-409e-b54a-91f4e6482108 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:36.092614 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf367f1-d316-4315-82ec-ac4bb87fb0b0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:36.104513 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb404ea0-7cb4-409e-b54a-91f4e6482108 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:36.118004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecea74d-74e9-4c39-911e-a444f5fda845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:36.118619 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecea74d-74e9-4c39-911e-a444f5fda845 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:36.119073 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecea74d-74e9-4c39-911e-a444f5fda845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:36.119428 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecea74d-74e9-4c39-911e-a444f5fda845 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:36.119985 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecea74d-74e9-4c39-911e-a444f5fda845 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:02:36.120276 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecea74d-74e9-4c39-911e-a444f5fda845 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:36.121938 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecea74d-74e9-4c39-911e-a444f5fda845 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:36.138442 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:36.147087 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1009/2003] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:02:36.148403 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecea74d-74e9-4c39-911e-a444f5fda845 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:36.158611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecea74d-74e9-4c39-911e-a444f5fda845 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:36.163304 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e19644-1322-4670-8355-00628fc7bc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:36.164039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e19644-1322-4670-8355-00628fc7bc44 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:36.164385 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e19644-1322-4670-8355-00628fc7bc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:36.164702 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e19644-1322-4670-8355-00628fc7bc44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:36.165393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e19644-1322-4670-8355-00628fc7bc44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:36.167309 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e19644-1322-4670-8355-00628fc7bc44 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:36.250397 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:36.258114 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1010/2004] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:02:36 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:02:36.262246 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e19644-1322-4670-8355-00628fc7bc44 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:36.324262 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e19644-1322-4670-8355-00628fc7bc44 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:36.327755 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bdf367f1-d316-4315-82ec-ac4bb87fb0b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '640a5815b1a545fc952ac67f553b6797'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:36.329534 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6688916c-bd7e-4915-a7cb-65ad6ed09540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:36.330271 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688916c-bd7e-4915-a7cb-65ad6ed09540 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:36.330548 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688916c-bd7e-4915-a7cb-65ad6ed09540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:36.335044 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688916c-bd7e-4915-a7cb-65ad6ed09540 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:36.335044 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6688916c-bd7e-4915-a7cb-65ad6ed09540 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:36.335044 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6688916c-bd7e-4915-a7cb-65ad6ed09540 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:36.335301 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bdf367f1-d316-4315-82ec-ac4bb87fb0b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '640a5815b1a545fc952ac67f553b6797'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:36.347321 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bdf367f1-d316-4315-82ec-ac4bb87fb0b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '640a5815b1a545fc952ac67f553b6797'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:36.415673 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bdf367f1-d316-4315-82ec-ac4bb87fb0b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '640a5815b1a545fc952ac67f553b6797'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:36.416300 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-bdf367f1-d316-4315-82ec-ac4bb87fb0b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '640a5815b1a545fc952ac67f553b6797'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:36.421570 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1011/2005] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:02:36 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:02:36.422639 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 995/2006] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:02:35 2022] DELETE /identity/v3/users/640a5815b1a545fc952ac67f553b6797 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:36.426293 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6688916c-bd7e-4915-a7cb-65ad6ed09540 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:36.427569 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6688916c-bd7e-4915-a7cb-65ad6ed09540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:36.440081 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6688916c-bd7e-4915-a7cb-65ad6ed09540 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:36.502734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425c6d3-e900-4ef7-b60f-8f2f18f0eeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:36.503422 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425c6d3-e900-4ef7-b60f-8f2f18f0eeeb admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:36.503743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425c6d3-e900-4ef7-b60f-8f2f18f0eeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:36.504020 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425c6d3-e900-4ef7-b60f-8f2f18f0eeeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:36.504719 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8425c6d3-e900-4ef7-b60f-8f2f18f0eeeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:36.506365 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8425c6d3-e900-4ef7-b60f-8f2f18f0eeeb admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:36.516294 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1012/2007] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:36.521546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf367f1-d316-4315-82ec-ac4bb87fb0b0 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:36.534808 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf367f1-d316-4315-82ec-ac4bb87fb0b0 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:36.541041 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146e1b9-011b-4048-b771-9569f3ff229e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:36.541735 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146e1b9-011b-4048-b771-9569f3ff229e admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:36.542163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146e1b9-011b-4048-b771-9569f3ff229e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:36.542443 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146e1b9-011b-4048-b771-9569f3ff229e admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:36.543133 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146e1b9-011b-4048-b771-9569f3ff229e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:36.544810 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146e1b9-011b-4048-b771-9569f3ff229e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:36.556003 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 996/2008] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:36.560725 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425c6d3-e900-4ef7-b60f-8f2f18f0eeeb admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:36.572449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425c6d3-e900-4ef7-b60f-8f2f18f0eeeb admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:36.578191 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ad3fe-54b0-4ec9-bf18-f515a42e82b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:36.579300 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ad3fe-54b0-4ec9-bf18-f515a42e82b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:36.582037 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ad3fe-54b0-4ec9-bf18-f515a42e82b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:36.582415 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ad3fe-54b0-4ec9-bf18-f515a42e82b3 admin admin] PATH_INFO: `/v3/projects/21893cf5b64449f8a7d50a2856fc2dfd/users/fc208189b88e434ca09950f17dc8e1e2/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:36.611593 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ad3fe-54b0-4ec9-bf18-f515a42e82b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21893cf5b64449f8a7d50a2856fc2dfd, user_id=fc208189b88e434ca09950f17dc8e1e2, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:36.614242 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ad3fe-54b0-4ec9-bf18-f515a42e82b3 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:36.703210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146e1b9-011b-4048-b771-9569f3ff229e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:36.705589 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c146e1b9-011b-4048-b771-9569f3ff229e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:36.718602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146e1b9-011b-4048-b771-9569f3ff229e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:36.723838 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a79eca3-ae99-4b3c-aefd-03980ed0a5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:36.724469 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79eca3-ae99-4b3c-aefd-03980ed0a5b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:36.724746 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79eca3-ae99-4b3c-aefd-03980ed0a5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:36.725016 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79eca3-ae99-4b3c-aefd-03980ed0a5b4 admin admin] PATH_INFO: `/v3/projects/c748f4142cf54207a35d5d39b6cac877` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:36.726266 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a79eca3-ae99-4b3c-aefd-03980ed0a5b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c748f4142cf54207a35d5d39b6cac877)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:36.728292 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a79eca3-ae99-4b3c-aefd-03980ed0a5b4 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:36.743021 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1013/2009] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:36 2022] PUT /identity/v3/projects/21893cf5b64449f8a7d50a2856fc2dfd/users/fc208189b88e434ca09950f17dc8e1e2/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:02:36.753992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bd8bb-5543-44a3-93a1-5c0c595fac16 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:36.754306 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bd8bb-5543-44a3-93a1-5c0c595fac16 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:36.754575 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bd8bb-5543-44a3-93a1-5c0c595fac16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:36.782419 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-5b4bd8bb-5543-44a3-93a1-5c0c595fac16 None None] MFA Rules not processed for user `fc208189b88e434ca09950f17dc8e1e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:36.783842 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5b4bd8bb-5543-44a3-93a1-5c0c595fac16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:36.831204 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:36.839684 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0a79eca3-ae99-4b3c-aefd-03980ed0a5b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c748f4142cf54207a35d5d39b6cac877', 'request_id': 'req-0a79eca3-ae99-4b3c-aefd-03980ed0a5b4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:36.849102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1014/2010] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:02:36.857224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0a79eca3-ae99-4b3c-aefd-03980ed0a5b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c748f4142cf54207a35d5d39b6cac877', 'request_id': 'req-0a79eca3-ae99-4b3c-aefd-03980ed0a5b4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:36.860040 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4bd8bb-5543-44a3-93a1-5c0c595fac16 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:36.861960 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5b4bd8bb-5543-44a3-93a1-5c0c595fac16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:36.864035 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0a79eca3-ae99-4b3c-aefd-03980ed0a5b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c748f4142cf54207a35d5d39b6cac877', 'request_id': 'req-0a79eca3-ae99-4b3c-aefd-03980ed0a5b4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:36.919491 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4bd8bb-5543-44a3-93a1-5c0c595fac16 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:36.947203 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0a79eca3-ae99-4b3c-aefd-03980ed0a5b4 admin admin] The token cache is being invalidate because project c748f4142cf54207a35d5d39b6cac877 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:36.947408 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0a79eca3-ae99-4b3c-aefd-03980ed0a5b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:36.948094 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0a79eca3-ae99-4b3c-aefd-03980ed0a5b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:36.949689 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 997/2011] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:02:36 2022] DELETE /identity/v3/projects/c748f4142cf54207a35d5d39b6cac877 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:02:36.954487 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cda235-58ce-4ffc-988b-e72e0681b793 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:36.954960 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cda235-58ce-4ffc-988b-e72e0681b793 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:36.955413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cda235-58ce-4ffc-988b-e72e0681b793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:36.971189 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8e3d9-5d6f-4c55-bf96-4f38ae10145b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:36.971811 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8e3d9-5d6f-4c55-bf96-4f38ae10145b service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:36.972096 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8e3d9-5d6f-4c55-bf96-4f38ae10145b service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:36.972358 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8e3d9-5d6f-4c55-bf96-4f38ae10145b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:36.974205 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3ff8e3d9-5d6f-4c55-bf96-4f38ae10145b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:36.975975 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b7cda235-58ce-4ffc-988b-e72e0681b793 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:36.977180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b7cda235-58ce-4ffc-988b-e72e0681b793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:37.045789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 998/2012] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:37.052238 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8e3d9-5d6f-4c55-bf96-4f38ae10145b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:37.053672 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8e3d9-5d6f-4c55-bf96-4f38ae10145b service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:37.056078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e612a-bd3c-49c3-a797-a6d0c8dc31cb None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:37.056371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e612a-bd3c-49c3-a797-a6d0c8dc31cb None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:37.056649 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e612a-bd3c-49c3-a797-a6d0c8dc31cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:37.062920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1015/2013] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:02:37.076763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-234e612a-bd3c-49c3-a797-a6d0c8dc31cb None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:37.078019 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-234e612a-bd3c-49c3-a797-a6d0c8dc31cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:37.086862 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 999/2014] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:37.091275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8e3d9-5d6f-4c55-bf96-4f38ae10145b service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:37.102916 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8e3d9-5d6f-4c55-bf96-4f38ae10145b service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:37.107613 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd121b7e-5073-4835-81db-30d46a83f5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:37.108217 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd121b7e-5073-4835-81db-30d46a83f5d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:37.108497 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd121b7e-5073-4835-81db-30d46a83f5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:37.108770 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd121b7e-5073-4835-81db-30d46a83f5d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:37.109410 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd121b7e-5073-4835-81db-30d46a83f5d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:02:37.109712 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd121b7e-5073-4835-81db-30d46a83f5d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:37.111603 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd121b7e-5073-4835-81db-30d46a83f5d9 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:37.124630 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1016/2015] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:02:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:02:37.129083 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e612a-bd3c-49c3-a797-a6d0c8dc31cb None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:37.133555 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:37.138625 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e612a-bd3c-49c3-a797-a6d0c8dc31cb None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:37.144385 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d8cc3-2000-4c81-ade9-dce886bb87f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:37.145414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d8cc3-2000-4c81-ade9-dce886bb87f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:37.145880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d8cc3-2000-4c81-ade9-dce886bb87f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:37.146547 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d8cc3-2000-4c81-ade9-dce886bb87f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:37.147391 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d8cc3-2000-4c81-ade9-dce886bb87f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:37.149494 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d8cc3-2000-4c81-ade9-dce886bb87f7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:37.264023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1000/2016] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:02:37 2022] POST /identity/v3/projects => generated 365 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:02:37.268333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd121b7e-5073-4835-81db-30d46a83f5d9 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:37.278471 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd121b7e-5073-4835-81db-30d46a83f5d9 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:37.331097 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:37.344763 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc2876-72e9-4eff-b3fc-5badfe87a813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:37.345396 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc2876-72e9-4eff-b3fc-5badfe87a813 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:37.346174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc2876-72e9-4eff-b3fc-5badfe87a813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:37.346437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc2876-72e9-4eff-b3fc-5badfe87a813 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:37.347115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fc2876-72e9-4eff-b3fc-5badfe87a813 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:37.349025 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fc2876-72e9-4eff-b3fc-5badfe87a813 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:37.465561 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1017/2017] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:02:37 2022] POST /identity/v3/users => generated 283 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:37.469670 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d8cc3-2000-4c81-ade9-dce886bb87f7 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:37.479875 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d8cc3-2000-4c81-ade9-dce886bb87f7 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:37.484926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02b7b0-2db8-4fe8-af89-0693c7911794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:37.485519 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02b7b0-2db8-4fe8-af89-0693c7911794 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:37.487406 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02b7b0-2db8-4fe8-af89-0693c7911794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:37.487406 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02b7b0-2db8-4fe8-af89-0693c7911794 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:37.487406 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc02b7b0-2db8-4fe8-af89-0693c7911794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:37.488436 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc02b7b0-2db8-4fe8-af89-0693c7911794 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:37.498392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1001/2018] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:37.503107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc2876-72e9-4eff-b3fc-5badfe87a813 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:37.514346 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc2876-72e9-4eff-b3fc-5badfe87a813 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:37.519859 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af399e-a47f-450c-819b-416e23450a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:37.520452 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af399e-a47f-450c-819b-416e23450a33 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:37.520724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af399e-a47f-450c-819b-416e23450a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:37.520991 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af399e-a47f-450c-819b-416e23450a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:37.521643 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af399e-a47f-450c-819b-416e23450a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:37.523387 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af399e-a47f-450c-819b-416e23450a33 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:37.534564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1018/2019] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:37.538645 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02b7b0-2db8-4fe8-af89-0693c7911794 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:37.548396 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02b7b0-2db8-4fe8-af89-0693c7911794 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:37.553189 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe3ef1-a8f1-4441-b09d-da18eecb79c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:37.556454 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe3ef1-a8f1-4441-b09d-da18eecb79c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:37.556454 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe3ef1-a8f1-4441-b09d-da18eecb79c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:37.556454 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe3ef1-a8f1-4441-b09d-da18eecb79c6 admin admin] PATH_INFO: `/v3/projects/a2d527452f4a4a2088368039042fa7dd/users/9de6519722ba4804943751da63d1b45a/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:37.570506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe3ef1-a8f1-4441-b09d-da18eecb79c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2d527452f4a4a2088368039042fa7dd, user_id=9de6519722ba4804943751da63d1b45a, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:37.573241 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe3ef1-a8f1-4441-b09d-da18eecb79c6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:37.661702 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1002/2020] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:37 2022] PUT /identity/v3/projects/a2d527452f4a4a2088368039042fa7dd/users/9de6519722ba4804943751da63d1b45a/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:02:37.672195 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3beee-53c7-4b01-bc91-f26cb98db9f2 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:37.672572 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3beee-53c7-4b01-bc91-f26cb98db9f2 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:37.672967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3beee-53c7-4b01-bc91-f26cb98db9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:37.706930 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-8fe3beee-53c7-4b01-bc91-f26cb98db9f2 None None] MFA Rules not processed for user `9de6519722ba4804943751da63d1b45a`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:37.708653 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8fe3beee-53c7-4b01-bc91-f26cb98db9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:37.780942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1019/2021] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:37.792224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe3ef1-a8f1-4441-b09d-da18eecb79c6 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:37.796066 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a9fe3ef1-a8f1-4441-b09d-da18eecb79c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:37.855853 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe3ef1-a8f1-4441-b09d-da18eecb79c6 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:37.859612 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9078ad6-9485-4b27-a3cc-bc7a54cea550 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:37.860219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9078ad6-9485-4b27-a3cc-bc7a54cea550 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:37.860498 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9078ad6-9485-4b27-a3cc-bc7a54cea550 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:37.860770 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9078ad6-9485-4b27-a3cc-bc7a54cea550 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:37.869937 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9078ad6-9485-4b27-a3cc-bc7a54cea550 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:37.871435 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9078ad6-9485-4b27-a3cc-bc7a54cea550 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:37.880983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1003/2022] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:02:40.001499 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe3beee-53c7-4b01-bc91-f26cb98db9f2 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:40.003573 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8fe3beee-53c7-4b01-bc91-f26cb98db9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:40.068815 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe3beee-53c7-4b01-bc91-f26cb98db9f2 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:40.073263 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27d791-05b2-4a03-8d9c-96e9d8ec42e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:40.073888 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27d791-05b2-4a03-8d9c-96e9d8ec42e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:40.074166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27d791-05b2-4a03-8d9c-96e9d8ec42e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:40.074436 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27d791-05b2-4a03-8d9c-96e9d8ec42e9 admin admin] PATH_INFO: `/v3/users/5f26791a55834f7cacbef8c281e7deec` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:40.075583 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27d791-05b2-4a03-8d9c-96e9d8ec42e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f26791a55834f7cacbef8c281e7deec)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:40.077497 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27d791-05b2-4a03-8d9c-96e9d8ec42e9 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:40.300997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ac27d791-05b2-4a03-8d9c-96e9d8ec42e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f26791a55834f7cacbef8c281e7deec'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:40.309114 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:40.314737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ac27d791-05b2-4a03-8d9c-96e9d8ec42e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f26791a55834f7cacbef8c281e7deec'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:40.322480 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ac27d791-05b2-4a03-8d9c-96e9d8ec42e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f26791a55834f7cacbef8c281e7deec'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:40.322978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ac27d791-05b2-4a03-8d9c-96e9d8ec42e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f26791a55834f7cacbef8c281e7deec'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:40.399267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ac27d791-05b2-4a03-8d9c-96e9d8ec42e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f26791a55834f7cacbef8c281e7deec'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:40.405595 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1020/2023] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:02:40 2022] DELETE /identity/v3/users/5f26791a55834f7cacbef8c281e7deec => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:40.412449 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa37eb6-7bc8-4c29-9a1f-aa03f14a057d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:40.412904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa37eb6-7bc8-4c29-9a1f-aa03f14a057d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:40.412992 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa37eb6-7bc8-4c29-9a1f-aa03f14a057d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:40.432474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-baa37eb6-7bc8-4c29-9a1f-aa03f14a057d None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:40.433671 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-baa37eb6-7bc8-4c29-9a1f-aa03f14a057d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:40.497015 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1004/2024] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:40.507132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27d791-05b2-4a03-8d9c-96e9d8ec42e9 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:40.508563 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ac27d791-05b2-4a03-8d9c-96e9d8ec42e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:40.565925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27d791-05b2-4a03-8d9c-96e9d8ec42e9 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:40.569837 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9307aa-0154-45a6-8c47-32e6b02cfdf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:40.570399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9307aa-0154-45a6-8c47-32e6b02cfdf1 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:40.570703 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9307aa-0154-45a6-8c47-32e6b02cfdf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:40.570975 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9307aa-0154-45a6-8c47-32e6b02cfdf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:40.579944 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9307aa-0154-45a6-8c47-32e6b02cfdf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:40.581841 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9307aa-0154-45a6-8c47-32e6b02cfdf1 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:40.591176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1021/2025] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:02:40.993274 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa37eb6-7bc8-4c29-9a1f-aa03f14a057d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:40.994795 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-baa37eb6-7bc8-4c29-9a1f-aa03f14a057d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:41.008421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa37eb6-7bc8-4c29-9a1f-aa03f14a057d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:41.014069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3d3ab-9f3f-4323-aab1-f85c91134dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:41.014628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3d3ab-9f3f-4323-aab1-f85c91134dbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:41.015255 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3d3ab-9f3f-4323-aab1-f85c91134dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:41.015527 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3d3ab-9f3f-4323-aab1-f85c91134dbd admin admin] PATH_INFO: `/v3/projects/3bb6658f278f43b69b9d1a741fc2f82e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:41.016841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3d3ab-9f3f-4323-aab1-f85c91134dbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bb6658f278f43b69b9d1a741fc2f82e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:41.019495 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3d3ab-9f3f-4323-aab1-f85c91134dbd admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:41.145665 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:41.149365 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-74f3d3ab-9f3f-4323-aab1-f85c91134dbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bb6658f278f43b69b9d1a741fc2f82e', 'request_id': 'req-74f3d3ab-9f3f-4323-aab1-f85c91134dbd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:41.155835 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-74f3d3ab-9f3f-4323-aab1-f85c91134dbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bb6658f278f43b69b9d1a741fc2f82e', 'request_id': 'req-74f3d3ab-9f3f-4323-aab1-f85c91134dbd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:41.161470 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-74f3d3ab-9f3f-4323-aab1-f85c91134dbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bb6658f278f43b69b9d1a741fc2f82e', 'request_id': 'req-74f3d3ab-9f3f-4323-aab1-f85c91134dbd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:41.270269 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-74f3d3ab-9f3f-4323-aab1-f85c91134dbd admin admin] The token cache is being invalidate because project 3bb6658f278f43b69b9d1a741fc2f82e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:41.270553 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-74f3d3ab-9f3f-4323-aab1-f85c91134dbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:41.271229 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-74f3d3ab-9f3f-4323-aab1-f85c91134dbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:41.273029 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1005/2026] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:02:40 2022] DELETE /identity/v3/projects/3bb6658f278f43b69b9d1a741fc2f82e => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:41.276869 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9307aa-0154-45a6-8c47-32e6b02cfdf1 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:41.286537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9307aa-0154-45a6-8c47-32e6b02cfdf1 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:41.344226 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be8b44-ee8f-4361-9e3f-3c480c362d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:41.344846 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be8b44-ee8f-4361-9e3f-3c480c362d81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:41.345117 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be8b44-ee8f-4361-9e3f-3c480c362d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:41.345383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be8b44-ee8f-4361-9e3f-3c480c362d81 admin admin] PATH_INFO: `/v3/users/1ee9866bb84745cbbf515acf5e814113` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:41.346584 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be8b44-ee8f-4361-9e3f-3c480c362d81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ee9866bb84745cbbf515acf5e814113)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:41.348509 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be8b44-ee8f-4361-9e3f-3c480c362d81 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:41.484913 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-38be8b44-ee8f-4361-9e3f-3c480c362d81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ee9866bb84745cbbf515acf5e814113'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:41.497945 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-38be8b44-ee8f-4361-9e3f-3c480c362d81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ee9866bb84745cbbf515acf5e814113'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:41.505801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-38be8b44-ee8f-4361-9e3f-3c480c362d81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ee9866bb84745cbbf515acf5e814113'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:41.506481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-38be8b44-ee8f-4361-9e3f-3c480c362d81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ee9866bb84745cbbf515acf5e814113'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:41.510119 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:41.592489 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-38be8b44-ee8f-4361-9e3f-3c480c362d81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ee9866bb84745cbbf515acf5e814113'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:41.598859 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1022/2027] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:02:41 2022] DELETE /identity/v3/users/1ee9866bb84745cbbf515acf5e814113 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:41.999136 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3d3ab-9f3f-4323-aab1-f85c91134dbd admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:42.000642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-74f3d3ab-9f3f-4323-aab1-f85c91134dbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:42.015320 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3d3ab-9f3f-4323-aab1-f85c91134dbd admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:42.075523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daada51-1690-451b-9168-78cd0ab953b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:42.076151 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daada51-1690-451b-9168-78cd0ab953b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:42.076450 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daada51-1690-451b-9168-78cd0ab953b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:42.076815 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daada51-1690-451b-9168-78cd0ab953b0 admin admin] PATH_INFO: `/v3/projects/74567b6cf5f3460eac8de09251dc358c` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:42.077953 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daada51-1690-451b-9168-78cd0ab953b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74567b6cf5f3460eac8de09251dc358c)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:42.080022 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daada51-1690-451b-9168-78cd0ab953b0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:42.217297 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2daada51-1690-451b-9168-78cd0ab953b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74567b6cf5f3460eac8de09251dc358c', 'request_id': 'req-2daada51-1690-451b-9168-78cd0ab953b0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:42.223590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2daada51-1690-451b-9168-78cd0ab953b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74567b6cf5f3460eac8de09251dc358c', 'request_id': 'req-2daada51-1690-451b-9168-78cd0ab953b0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:42.229280 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2daada51-1690-451b-9168-78cd0ab953b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74567b6cf5f3460eac8de09251dc358c', 'request_id': 'req-2daada51-1690-451b-9168-78cd0ab953b0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:42.323986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2daada51-1690-451b-9168-78cd0ab953b0 admin admin] The token cache is being invalidate because project 74567b6cf5f3460eac8de09251dc358c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:42.324274 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2daada51-1690-451b-9168-78cd0ab953b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:42.325011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2daada51-1690-451b-9168-78cd0ab953b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:42.326489 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1006/2028] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:02:41 2022] DELETE /identity/v3/projects/74567b6cf5f3460eac8de09251dc358c => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:42.330989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be8b44-ee8f-4361-9e3f-3c480c362d81 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:42.332594 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-38be8b44-ee8f-4361-9e3f-3c480c362d81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:42.345743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be8b44-ee8f-4361-9e3f-3c480c362d81 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:42.403941 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c4918-18d3-459e-889c-d5301f24d6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:42.404583 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c4918-18d3-459e-889c-d5301f24d6ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:42.404858 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c4918-18d3-459e-889c-d5301f24d6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:42.405132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c4918-18d3-459e-889c-d5301f24d6ba admin admin] PATH_INFO: `/v3/users/29399bab1ac347b4a546bb84eca6d05a` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:42.406273 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c4918-18d3-459e-889c-d5301f24d6ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=29399bab1ac347b4a546bb84eca6d05a)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:42.408240 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c4918-18d3-459e-889c-d5301f24d6ba admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:42.629200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c20c4918-18d3-459e-889c-d5301f24d6ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29399bab1ac347b4a546bb84eca6d05a'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:42.641795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c20c4918-18d3-459e-889c-d5301f24d6ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29399bab1ac347b4a546bb84eca6d05a'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:42.649353 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c20c4918-18d3-459e-889c-d5301f24d6ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29399bab1ac347b4a546bb84eca6d05a'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:42.650038 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c20c4918-18d3-459e-889c-d5301f24d6ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29399bab1ac347b4a546bb84eca6d05a'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:42.729934 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c20c4918-18d3-459e-889c-d5301f24d6ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29399bab1ac347b4a546bb84eca6d05a'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:42.736311 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1023/2029] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:02:42 2022] DELETE /identity/v3/users/29399bab1ac347b4a546bb84eca6d05a => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:43.160642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daada51-1690-451b-9168-78cd0ab953b0 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:43.162196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2daada51-1690-451b-9168-78cd0ab953b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:43.183630 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daada51-1690-451b-9168-78cd0ab953b0 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:43.262446 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb262f-e947-4595-a45d-5caba4a1571c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:43.263434 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb262f-e947-4595-a45d-5caba4a1571c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:43.263705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb262f-e947-4595-a45d-5caba4a1571c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:43.264030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb262f-e947-4595-a45d-5caba4a1571c admin admin] PATH_INFO: `/v3/projects/a7245f20608948f7b9c383d9271cce2b` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:43.265103 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebb262f-e947-4595-a45d-5caba4a1571c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7245f20608948f7b9c383d9271cce2b)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:43.267468 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebb262f-e947-4595-a45d-5caba4a1571c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:43.270623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:43.414764 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-eebb262f-e947-4595-a45d-5caba4a1571c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7245f20608948f7b9c383d9271cce2b', 'request_id': 'req-eebb262f-e947-4595-a45d-5caba4a1571c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:43.425146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-eebb262f-e947-4595-a45d-5caba4a1571c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7245f20608948f7b9c383d9271cce2b', 'request_id': 'req-eebb262f-e947-4595-a45d-5caba4a1571c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:43.433755 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-eebb262f-e947-4595-a45d-5caba4a1571c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7245f20608948f7b9c383d9271cce2b', 'request_id': 'req-eebb262f-e947-4595-a45d-5caba4a1571c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:43.529228 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-eebb262f-e947-4595-a45d-5caba4a1571c admin admin] The token cache is being invalidate because project a7245f20608948f7b9c383d9271cce2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:43.529439 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-eebb262f-e947-4595-a45d-5caba4a1571c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:43.530059 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-eebb262f-e947-4595-a45d-5caba4a1571c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:43.531576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1007/2030] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:02:43 2022] DELETE /identity/v3/projects/a7245f20608948f7b9c383d9271cce2b => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:43.537631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff9a35-d838-41d8-bf3e-82b7a1dede2f None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:43.538093 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff9a35-d838-41d8-bf3e-82b7a1dede2f None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:43.538180 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff9a35-d838-41d8-bf3e-82b7a1dede2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:43.558099 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-4bff9a35-d838-41d8-bf3e-82b7a1dede2f None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:43.562041 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4bff9a35-d838-41d8-bf3e-82b7a1dede2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:43.629319 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1024/2031] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:43.637583 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9191b-ef96-40cf-9e39-98bf4dbdbddb None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:43.637873 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9191b-ef96-40cf-9e39-98bf4dbdbddb None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:43.638151 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9191b-ef96-40cf-9e39-98bf4dbdbddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:43.657962 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b3e9191b-ef96-40cf-9e39-98bf4dbdbddb None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:43.659286 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b3e9191b-ef96-40cf-9e39-98bf4dbdbddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:43.667397 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1008/2032] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:43.671912 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff9a35-d838-41d8-bf3e-82b7a1dede2f None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:43.682227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff9a35-d838-41d8-bf3e-82b7a1dede2f None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:43.687060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa284ef0-32d2-49ea-9ca3-f37763266c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:43.687660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa284ef0-32d2-49ea-9ca3-f37763266c8b admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:43.687931 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa284ef0-32d2-49ea-9ca3-f37763266c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:43.688231 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa284ef0-32d2-49ea-9ca3-f37763266c8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:43.688863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa284ef0-32d2-49ea-9ca3-f37763266c8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:02:43.689158 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa284ef0-32d2-49ea-9ca3-f37763266c8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:43.691880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa284ef0-32d2-49ea-9ca3-f37763266c8b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:43.707875 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1025/2033] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:02:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:02:43.713757 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e9191b-ef96-40cf-9e39-98bf4dbdbddb None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:43.729613 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e9191b-ef96-40cf-9e39-98bf4dbdbddb None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:43.733713 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf2a7d-f4e3-4e46-8114-225f5583022c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:43.734339 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf2a7d-f4e3-4e46-8114-225f5583022c admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:43.734618 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf2a7d-f4e3-4e46-8114-225f5583022c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:43.734936 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf2a7d-f4e3-4e46-8114-225f5583022c admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:43.735571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf2a7d-f4e3-4e46-8114-225f5583022c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:43.738172 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf2a7d-f4e3-4e46-8114-225f5583022c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:43.746048 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa284ef0-32d2-49ea-9ca3-f37763266c8b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:43.747766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-aa284ef0-32d2-49ea-9ca3-f37763266c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:43.760512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:43.765944 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa284ef0-32d2-49ea-9ca3-f37763266c8b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:43.771245 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5939907-8c33-4ef9-a13d-81af86bbe70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:43.772136 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5939907-8c33-4ef9-a13d-81af86bbe70b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:43.772251 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5939907-8c33-4ef9-a13d-81af86bbe70b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:43.772448 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5939907-8c33-4ef9-a13d-81af86bbe70b admin admin] PATH_INFO: `/v3/users/fc208189b88e434ca09950f17dc8e1e2` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:43.773623 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5939907-8c33-4ef9-a13d-81af86bbe70b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc208189b88e434ca09950f17dc8e1e2)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:43.775553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5939907-8c33-4ef9-a13d-81af86bbe70b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:43.825074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1009/2034] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:02:43 2022] POST /identity/v3/projects => generated 367 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:43.834444 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf2a7d-f4e3-4e46-8114-225f5583022c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:43.899224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf2a7d-f4e3-4e46-8114-225f5583022c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:43.903875 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4468e0d-cd44-4864-9b95-96028d46a565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:43.904476 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4468e0d-cd44-4864-9b95-96028d46a565 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:43.904758 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4468e0d-cd44-4864-9b95-96028d46a565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:43.905034 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4468e0d-cd44-4864-9b95-96028d46a565 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:43.905662 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4468e0d-cd44-4864-9b95-96028d46a565 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:43.907640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4468e0d-cd44-4864-9b95-96028d46a565 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:43.984287 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f5939907-8c33-4ef9-a13d-81af86bbe70b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc208189b88e434ca09950f17dc8e1e2'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:43.996286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f5939907-8c33-4ef9-a13d-81af86bbe70b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc208189b88e434ca09950f17dc8e1e2'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:44.004378 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f5939907-8c33-4ef9-a13d-81af86bbe70b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc208189b88e434ca09950f17dc8e1e2'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:44.005059 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f5939907-8c33-4ef9-a13d-81af86bbe70b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc208189b88e434ca09950f17dc8e1e2'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:44.005756 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1010/2035] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:02:43 2022] POST /identity/v3/users => generated 284 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:02:44.006603 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4468e0d-cd44-4864-9b95-96028d46a565 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:44.008260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d4468e0d-cd44-4864-9b95-96028d46a565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:44.021521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4468e0d-cd44-4864-9b95-96028d46a565 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:44.026564 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ccd839-a868-426a-996d-e51d360c4e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:44.027334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccd839-a868-426a-996d-e51d360c4e26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:44.027615 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccd839-a868-426a-996d-e51d360c4e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:44.033680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccd839-a868-426a-996d-e51d360c4e26 admin admin] PATH_INFO: `/v3/users/7eade3fb48284b088a1b2ce5e74e9383` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:44.033680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ccd839-a868-426a-996d-e51d360c4e26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7eade3fb48284b088a1b2ce5e74e9383)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:44.033680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ccd839-a868-426a-996d-e51d360c4e26 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:44.073936 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f5939907-8c33-4ef9-a13d-81af86bbe70b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc208189b88e434ca09950f17dc8e1e2'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:44.080919 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1026/2036] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:02:43 2022] DELETE /identity/v3/users/fc208189b88e434ca09950f17dc8e1e2 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:02:44.080919 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5939907-8c33-4ef9-a13d-81af86bbe70b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:44.083142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f5939907-8c33-4ef9-a13d-81af86bbe70b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:44.156416 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5939907-8c33-4ef9-a13d-81af86bbe70b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:44.161616 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17952b-9388-4505-821e-50a3b6208400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:44.162223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17952b-9388-4505-821e-50a3b6208400 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:44.162500 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17952b-9388-4505-821e-50a3b6208400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:44.162855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17952b-9388-4505-821e-50a3b6208400 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:44.163492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17952b-9388-4505-821e-50a3b6208400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:44.165143 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17952b-9388-4505-821e-50a3b6208400 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:44.175338 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1027/2037] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:44.176648 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1b413-9a6e-4059-9921-a7fa28314a05 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:44.176932 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1b413-9a6e-4059-9921-a7fa28314a05 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:44.177198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1b413-9a6e-4059-9921-a7fa28314a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:44.197295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-2ca1b413-9a6e-4059-9921-a7fa28314a05 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:44.198570 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2ca1b413-9a6e-4059-9921-a7fa28314a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:44.207911 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1028/2038] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:44.208830 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1b413-9a6e-4059-9921-a7fa28314a05 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:44.218169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1b413-9a6e-4059-9921-a7fa28314a05 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:44.223337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227b189-4737-4e60-af87-ac8b6fd346c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:44.223919 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227b189-4737-4e60-af87-ac8b6fd346c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:44.224185 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227b189-4737-4e60-af87-ac8b6fd346c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:44.224446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227b189-4737-4e60-af87-ac8b6fd346c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:44.225076 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c227b189-4737-4e60-af87-ac8b6fd346c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:44.226645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c227b189-4737-4e60-af87-ac8b6fd346c8 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:44.236328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1029/2039] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:44.237414 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227b189-4737-4e60-af87-ac8b6fd346c8 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:44.239124 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c227b189-4737-4e60-af87-ac8b6fd346c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:44.294585 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227b189-4737-4e60-af87-ac8b6fd346c8 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:44.298415 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62431364-0cca-4c87-aaf5-4dcc79e19c71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:44.299046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62431364-0cca-4c87-aaf5-4dcc79e19c71 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:44.299391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62431364-0cca-4c87-aaf5-4dcc79e19c71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:44.299925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62431364-0cca-4c87-aaf5-4dcc79e19c71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:44.310390 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62431364-0cca-4c87-aaf5-4dcc79e19c71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:44.311797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62431364-0cca-4c87-aaf5-4dcc79e19c71 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:44.321863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1030/2040] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:02:44.322614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62431364-0cca-4c87-aaf5-4dcc79e19c71 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:44.332445 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62431364-0cca-4c87-aaf5-4dcc79e19c71 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:44.337167 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f01160-6470-4477-a324-56db9d7064ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:44.337808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f01160-6470-4477-a324-56db9d7064ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:44.338089 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f01160-6470-4477-a324-56db9d7064ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:44.338364 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f01160-6470-4477-a324-56db9d7064ab admin admin] PATH_INFO: `/v3/projects/1316a89a9bc7405890697a5e1cd96927/users/dc4f11207b6046ffb56ccdb4e63103b5/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:44.353158 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f01160-6470-4477-a324-56db9d7064ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=1316a89a9bc7405890697a5e1cd96927, user_id=dc4f11207b6046ffb56ccdb4e63103b5, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:44.356915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f01160-6470-4477-a324-56db9d7064ab admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:44.416238 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:44.467611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1031/2041] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:44 2022] PUT /identity/v3/projects/1316a89a9bc7405890697a5e1cd96927/users/dc4f11207b6046ffb56ccdb4e63103b5/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:44.483276 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884e486-f316-4a40-bc9e-3a8f4cd7099e None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:44.483560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884e486-f316-4a40-bc9e-3a8f4cd7099e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:44.483838 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884e486-f316-4a40-bc9e-3a8f4cd7099e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:44.508758 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-61ccd839-a868-426a-996d-e51d360c4e26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7eade3fb48284b088a1b2ce5e74e9383'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:44.510715 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=81,keystone:DELETE=14,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:44.519527 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-61ccd839-a868-426a-996d-e51d360c4e26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7eade3fb48284b088a1b2ce5e74e9383'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:44.535724 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-61ccd839-a868-426a-996d-e51d360c4e26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7eade3fb48284b088a1b2ce5e74e9383'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:44.549736 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a884e486-f316-4a40-bc9e-3a8f4cd7099e None None] MFA Rules not processed for user `dc4f11207b6046ffb56ccdb4e63103b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:44.551825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a884e486-f316-4a40-bc9e-3a8f4cd7099e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:44.631675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-61ccd839-a868-426a-996d-e51d360c4e26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7eade3fb48284b088a1b2ce5e74e9383'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:44.632292 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-61ccd839-a868-426a-996d-e51d360c4e26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7eade3fb48284b088a1b2ce5e74e9383'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:44.642317 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1032/2042] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:02:44.649303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1011/2043] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:02:44 2022] DELETE /identity/v3/users/7eade3fb48284b088a1b2ce5e74e9383 => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:44.649454 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884e486-f316-4a40-bc9e-3a8f4cd7099e None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:44.652734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97ddaf-7e0d-485c-a614-d0560e3f7b94 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:44.653023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97ddaf-7e0d-485c-a614-d0560e3f7b94 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:44.653293 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97ddaf-7e0d-485c-a614-d0560e3f7b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:44.677684 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0c97ddaf-7e0d-485c-a614-d0560e3f7b94 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:44.680004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0c97ddaf-7e0d-485c-a614-d0560e3f7b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:44.741824 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884e486-f316-4a40-bc9e-3a8f4cd7099e None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:44.782097 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1012/2044] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:44.792073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97ddaf-7e0d-485c-a614-d0560e3f7b94 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:44.793502 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0c97ddaf-7e0d-485c-a614-d0560e3f7b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:44.808082 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbcc6dd-f4c2-4546-b837-6959f387a62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:44.808773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcc6dd-f4c2-4546-b837-6959f387a62a admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:44.809036 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcc6dd-f4c2-4546-b837-6959f387a62a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:44.809678 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcc6dd-f4c2-4546-b837-6959f387a62a admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:44.810324 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbcc6dd-f4c2-4546-b837-6959f387a62a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:44.812490 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbcc6dd-f4c2-4546-b837-6959f387a62a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:44.863956 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97ddaf-7e0d-485c-a614-d0560e3f7b94 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:44.867567 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73989d50-7e77-4662-a870-304c73635eaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:44.868199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73989d50-7e77-4662-a870-304c73635eaa service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:44.868476 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73989d50-7e77-4662-a870-304c73635eaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:44.868744 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73989d50-7e77-4662-a870-304c73635eaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:44.878204 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73989d50-7e77-4662-a870-304c73635eaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:44.879636 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73989d50-7e77-4662-a870-304c73635eaa service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:44.889807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1013/2045] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:02:44.891326 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73989d50-7e77-4662-a870-304c73635eaa service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:44.892686 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-73989d50-7e77-4662-a870-304c73635eaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:44.907634 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73989d50-7e77-4662-a870-304c73635eaa service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:44.919884 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc1b26a-f394-4d8d-be08-6b9f7c76671d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:44.919884 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1b26a-f394-4d8d-be08-6b9f7c76671d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:44.919884 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1b26a-f394-4d8d-be08-6b9f7c76671d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:44.919884 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1b26a-f394-4d8d-be08-6b9f7c76671d admin admin] PATH_INFO: `/v3/projects/21893cf5b64449f8a7d50a2856fc2dfd` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:44.920231 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc1b26a-f394-4d8d-be08-6b9f7c76671d admin admin] RBAC: Authorizing `identity:delete_project(project_id=21893cf5b64449f8a7d50a2856fc2dfd)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:44.920231 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc1b26a-f394-4d8d-be08-6b9f7c76671d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:44.954407 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1033/2046] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:02:44 2022] POST /identity/v3/projects => generated 367 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:44.958550 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbcc6dd-f4c2-4546-b837-6959f387a62a admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:44.962998 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8fbcc6dd-f4c2-4546-b837-6959f387a62a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:44.979364 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:45.040100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbcc6dd-f4c2-4546-b837-6959f387a62a admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:45.045249 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd180e-ca22-4c69-bf53-bbfa392041d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:45.045865 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd180e-ca22-4c69-bf53-bbfa392041d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:45.046140 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd180e-ca22-4c69-bf53-bbfa392041d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:45.046405 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd180e-ca22-4c69-bf53-bbfa392041d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:45.047159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd180e-ca22-4c69-bf53-bbfa392041d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:45.049689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd180e-ca22-4c69-bf53-bbfa392041d4 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:45.059916 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8bc1b26a-f394-4d8d-be08-6b9f7c76671d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21893cf5b64449f8a7d50a2856fc2dfd', 'request_id': 'req-8bc1b26a-f394-4d8d-be08-6b9f7c76671d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:45.070567 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8bc1b26a-f394-4d8d-be08-6b9f7c76671d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21893cf5b64449f8a7d50a2856fc2dfd', 'request_id': 'req-8bc1b26a-f394-4d8d-be08-6b9f7c76671d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:45.072641 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8bc1b26a-f394-4d8d-be08-6b9f7c76671d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21893cf5b64449f8a7d50a2856fc2dfd', 'request_id': 'req-8bc1b26a-f394-4d8d-be08-6b9f7c76671d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:45.159686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1034/2047] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:02:44 2022] POST /identity/v3/users => generated 290 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:45.163527 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd180e-ca22-4c69-bf53-bbfa392041d4 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:45.170364 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:45.175673 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd180e-ca22-4c69-bf53-bbfa392041d4 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:45.178868 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8bc1b26a-f394-4d8d-be08-6b9f7c76671d admin admin] The token cache is being invalidate because project 21893cf5b64449f8a7d50a2856fc2dfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:45.179573 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8bc1b26a-f394-4d8d-be08-6b9f7c76671d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:45.180323 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8bc1b26a-f394-4d8d-be08-6b9f7c76671d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:45.182114 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1014/2048] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:02:44 2022] DELETE /identity/v3/projects/21893cf5b64449f8a7d50a2856fc2dfd => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:45.187002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff99d75-2a24-483b-909c-d9c50881df4c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:45.187289 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff99d75-2a24-483b-909c-d9c50881df4c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:45.187569 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff99d75-2a24-483b-909c-d9c50881df4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:45.211167 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1ff99d75-2a24-483b-909c-d9c50881df4c None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:45.212407 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1ff99d75-2a24-483b-909c-d9c50881df4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:45.242277 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f85d2-5566-47e8-9fea-c4fdc1997cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:45.243293 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f85d2-5566-47e8-9fea-c4fdc1997cea admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:45.245530 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f85d2-5566-47e8-9fea-c4fdc1997cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:45.245530 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f85d2-5566-47e8-9fea-c4fdc1997cea admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:45.245530 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f85d2-5566-47e8-9fea-c4fdc1997cea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:45.247227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f85d2-5566-47e8-9fea-c4fdc1997cea admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:45.270792 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1035/2049] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:02:45.272128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f85d2-5566-47e8-9fea-c4fdc1997cea admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:45.273917 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b73f85d2-5566-47e8-9fea-c4fdc1997cea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:45.284367 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1015/2050] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:02:45.287050 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f85d2-5566-47e8-9fea-c4fdc1997cea admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:45.291876 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b1dda-bfa2-4308-8c1b-b32b1a610452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:45.292231 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbb311-dd7e-4ebc-b6af-186bfa677d81 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:45.292501 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbb311-dd7e-4ebc-b6af-186bfa677d81 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:45.292602 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b1dda-bfa2-4308-8c1b-b32b1a610452 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:45.292772 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbb311-dd7e-4ebc-b6af-186bfa677d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:45.292870 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b1dda-bfa2-4308-8c1b-b32b1a610452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:45.293036 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b1dda-bfa2-4308-8c1b-b32b1a610452 admin admin] PATH_INFO: `/v3/projects/5ce0d0e814d94907bea01839a8914b72/users/5850ffa4a1544b2cbe7a81b338f2537f/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:45.308467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b1dda-bfa2-4308-8c1b-b32b1a610452 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ce0d0e814d94907bea01839a8914b72, user_id=5850ffa4a1544b2cbe7a81b338f2537f, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:45.311425 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b1dda-bfa2-4308-8c1b-b32b1a610452 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:45.320224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-65dbb311-dd7e-4ebc-b6af-186bfa677d81 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:45.321494 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-65dbb311-dd7e-4ebc-b6af-186bfa677d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:45.329208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1016/2051] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:02:45.333915 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbb311-dd7e-4ebc-b6af-186bfa677d81 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:45.345279 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbb311-dd7e-4ebc-b6af-186bfa677d81 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:45.352195 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59451636-a451-47b4-9731-0d9d1a1fe152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:45.352731 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59451636-a451-47b4-9731-0d9d1a1fe152 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:45.353009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59451636-a451-47b4-9731-0d9d1a1fe152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:45.353278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59451636-a451-47b4-9731-0d9d1a1fe152 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:45.353922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59451636-a451-47b4-9731-0d9d1a1fe152 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:02:45.354221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59451636-a451-47b4-9731-0d9d1a1fe152 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:45.355896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59451636-a451-47b4-9731-0d9d1a1fe152 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:45.372401 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1017/2052] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:02:45.374159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59451636-a451-47b4-9731-0d9d1a1fe152 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:45.376256 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-59451636-a451-47b4-9731-0d9d1a1fe152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:45.389657 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59451636-a451-47b4-9731-0d9d1a1fe152 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:45.395506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d610aa-017b-44da-800d-0f15d8bc8180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:45.396125 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d610aa-017b-44da-800d-0f15d8bc8180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:45.396401 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d610aa-017b-44da-800d-0f15d8bc8180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:45.396674 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d610aa-017b-44da-800d-0f15d8bc8180 admin admin] PATH_INFO: `/v3/projects/d2029a9d5a294561b55de8901348554e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:45.397819 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d610aa-017b-44da-800d-0f15d8bc8180 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2029a9d5a294561b55de8901348554e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:45.399759 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d610aa-017b-44da-800d-0f15d8bc8180 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:45.403999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1036/2053] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:45 2022] PUT /identity/v3/projects/5ce0d0e814d94907bea01839a8914b72/users/5850ffa4a1544b2cbe7a81b338f2537f/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:02:45.403999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b1dda-bfa2-4308-8c1b-b32b1a610452 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:45.469246 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b1dda-bfa2-4308-8c1b-b32b1a610452 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:45.474679 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdeb54e-4201-4a58-90b3-59a46299438a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:45.477118 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdeb54e-4201-4a58-90b3-59a46299438a admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:45.477118 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdeb54e-4201-4a58-90b3-59a46299438a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:45.477118 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdeb54e-4201-4a58-90b3-59a46299438a admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:45.477118 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdeb54e-4201-4a58-90b3-59a46299438a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:45.478359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdeb54e-4201-4a58-90b3-59a46299438a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:45.550405 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:45.551020 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-24d610aa-017b-44da-800d-0f15d8bc8180 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2029a9d5a294561b55de8901348554e', 'request_id': 'req-24d610aa-017b-44da-800d-0f15d8bc8180'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:45.553183 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-24d610aa-017b-44da-800d-0f15d8bc8180 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2029a9d5a294561b55de8901348554e', 'request_id': 'req-24d610aa-017b-44da-800d-0f15d8bc8180'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:45.558923 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-24d610aa-017b-44da-800d-0f15d8bc8180 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2029a9d5a294561b55de8901348554e', 'request_id': 'req-24d610aa-017b-44da-800d-0f15d8bc8180'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:45.602337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1037/2054] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:02:45 2022] POST /identity/v3/projects => generated 367 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:45.602337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdeb54e-4201-4a58-90b3-59a46299438a admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:45.624870 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-24d610aa-017b-44da-800d-0f15d8bc8180 admin admin] The token cache is being invalidate because project d2029a9d5a294561b55de8901348554e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:45.624870 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-24d610aa-017b-44da-800d-0f15d8bc8180 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:45.624870 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-24d610aa-017b-44da-800d-0f15d8bc8180 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:45.625257 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1018/2055] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:02:45 2022] DELETE /identity/v3/projects/d2029a9d5a294561b55de8901348554e => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:45.626098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d610aa-017b-44da-800d-0f15d8bc8180 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:45.627521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-24d610aa-017b-44da-800d-0f15d8bc8180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:45.671220 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdeb54e-4201-4a58-90b3-59a46299438a admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:45.677210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34e268-c317-4bea-9eaf-6b503dc8bffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:45.677817 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34e268-c317-4bea-9eaf-6b503dc8bffd admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:45.678091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34e268-c317-4bea-9eaf-6b503dc8bffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:45.678363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34e268-c317-4bea-9eaf-6b503dc8bffd admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:45.679041 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb34e268-c317-4bea-9eaf-6b503dc8bffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:45.680640 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb34e268-c317-4bea-9eaf-6b503dc8bffd admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:45.694128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1038/2056] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:45.694128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34e268-c317-4bea-9eaf-6b503dc8bffd admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:45.694128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cb34e268-c317-4bea-9eaf-6b503dc8bffd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:45.699635 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d610aa-017b-44da-800d-0f15d8bc8180 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:45.703341 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7199b-b61c-47dc-8a50-8a46d7a5ffd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:45.704013 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7199b-b61c-47dc-8a50-8a46d7a5ffd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:45.704293 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7199b-b61c-47dc-8a50-8a46d7a5ffd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:45.704653 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7199b-b61c-47dc-8a50-8a46d7a5ffd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:45.705494 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e7199b-b61c-47dc-8a50-8a46d7a5ffd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:45.707489 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e7199b-b61c-47dc-8a50-8a46d7a5ffd7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:45.708268 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34e268-c317-4bea-9eaf-6b503dc8bffd admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:45.714124 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767a87b-cb76-4701-aff5-19a1d7684009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:45.714787 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767a87b-cb76-4701-aff5-19a1d7684009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:45.715063 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767a87b-cb76-4701-aff5-19a1d7684009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:45.715330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767a87b-cb76-4701-aff5-19a1d7684009 admin admin] PATH_INFO: `/v3/users/5a444a2c54d147bc857efc59c2bb29bb` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:45.716722 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3767a87b-cb76-4701-aff5-19a1d7684009 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a444a2c54d147bc857efc59c2bb29bb)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:45.720733 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3767a87b-cb76-4701-aff5-19a1d7684009 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:45.836642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1019/2057] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:02:45 2022] POST /identity/v3/users => generated 284 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:45.836642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7199b-b61c-47dc-8a50-8a46d7a5ffd7 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:45.839070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-90e7199b-b61c-47dc-8a50-8a46d7a5ffd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:45.857490 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7199b-b61c-47dc-8a50-8a46d7a5ffd7 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:45.865270 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6ebe9-fb46-42fa-960b-94cf5ae7f538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:45.866056 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6ebe9-fb46-42fa-960b-94cf5ae7f538 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:45.866552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6ebe9-fb46-42fa-960b-94cf5ae7f538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:45.867031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6ebe9-fb46-42fa-960b-94cf5ae7f538 admin admin] PATH_INFO: `/v3/projects/5ce0d0e814d94907bea01839a8914b72/users/5850ffa4a1544b2cbe7a81b338f2537f/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:45.869705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6ebe9-fb46-42fa-960b-94cf5ae7f538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ce0d0e814d94907bea01839a8914b72, user_id=5850ffa4a1544b2cbe7a81b338f2537f, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:45.873944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6ebe9-fb46-42fa-960b-94cf5ae7f538 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:45.897630 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1020/2058] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:45 2022] PUT /identity/v3/projects/5ce0d0e814d94907bea01839a8914b72/users/5850ffa4a1544b2cbe7a81b338f2537f/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:45.897630 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6ebe9-fb46-42fa-960b-94cf5ae7f538 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:45.918247 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-3767a87b-cb76-4701-aff5-19a1d7684009 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a444a2c54d147bc857efc59c2bb29bb'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:45.933531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-3767a87b-cb76-4701-aff5-19a1d7684009 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a444a2c54d147bc857efc59c2bb29bb'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:45.943376 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-3767a87b-cb76-4701-aff5-19a1d7684009 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a444a2c54d147bc857efc59c2bb29bb'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:45.944391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-3767a87b-cb76-4701-aff5-19a1d7684009 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a444a2c54d147bc857efc59c2bb29bb'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:45.978365 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6ebe9-fb46-42fa-960b-94cf5ae7f538 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:45.982958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39757d9b-161c-4de7-9775-12c53669e98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:45.990443 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39757d9b-161c-4de7-9775-12c53669e98b admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:45.990443 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39757d9b-161c-4de7-9775-12c53669e98b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:45.990443 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39757d9b-161c-4de7-9775-12c53669e98b admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:45.990443 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39757d9b-161c-4de7-9775-12c53669e98b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:45.990443 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39757d9b-161c-4de7-9775-12c53669e98b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:45.995939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1021/2059] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:45.997266 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23af36-355a-4412-9444-a308db5c5d5f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:45.997545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23af36-355a-4412-9444-a308db5c5d5f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:45.997815 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23af36-355a-4412-9444-a308db5c5d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:46.026952 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-6f23af36-355a-4412-9444-a308db5c5d5f None None] MFA Rules not processed for user `5850ffa4a1544b2cbe7a81b338f2537f`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:46.028234 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6f23af36-355a-4412-9444-a308db5c5d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:46.038451 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-3767a87b-cb76-4701-aff5-19a1d7684009 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a444a2c54d147bc857efc59c2bb29bb'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:46.044913 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1039/2060] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:02:45 2022] DELETE /identity/v3/users/5a444a2c54d147bc857efc59c2bb29bb => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:46.049500 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767a87b-cb76-4701-aff5-19a1d7684009 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:46.049500 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3767a87b-cb76-4701-aff5-19a1d7684009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:46.079358 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:46.109405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1022/2061] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:45 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:46.122094 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23af36-355a-4412-9444-a308db5c5d5f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:46.123645 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6f23af36-355a-4412-9444-a308db5c5d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:46.125543 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767a87b-cb76-4701-aff5-19a1d7684009 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:46.131852 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafe23f-1734-4ea9-879f-17cd856ca7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:46.132470 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafe23f-1734-4ea9-879f-17cd856ca7c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:46.132744 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafe23f-1734-4ea9-879f-17cd856ca7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:46.133182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafe23f-1734-4ea9-879f-17cd856ca7c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:46.133831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eafe23f-1734-4ea9-879f-17cd856ca7c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:46.136515 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eafe23f-1734-4ea9-879f-17cd856ca7c2 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:46.168440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1040/2062] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:46.168440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafe23f-1734-4ea9-879f-17cd856ca7c2 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:46.179333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafe23f-1734-4ea9-879f-17cd856ca7c2 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:46.184178 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2249b-040d-4ba2-85ef-62d081759986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:46.184825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2249b-040d-4ba2-85ef-62d081759986 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:46.185101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2249b-040d-4ba2-85ef-62d081759986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:46.185368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2249b-040d-4ba2-85ef-62d081759986 admin admin] PATH_INFO: `/v3/projects/96a4c885eada487d849017e4e6e981eb/users/255694a8bcd041eabb35733448eed42d/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:46.189770 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23af36-355a-4412-9444-a308db5c5d5f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:46.201584 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e2249b-040d-4ba2-85ef-62d081759986 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96a4c885eada487d849017e4e6e981eb, user_id=255694a8bcd041eabb35733448eed42d, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:46.209175 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e2249b-040d-4ba2-85ef-62d081759986 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:46.244443 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb41a5-090d-48f4-9ef6-f79f23b3ea9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:46.245061 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb41a5-090d-48f4-9ef6-f79f23b3ea9a service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:46.245336 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb41a5-090d-48f4-9ef6-f79f23b3ea9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:46.245598 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb41a5-090d-48f4-9ef6-f79f23b3ea9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:46.279358 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1041/2063] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:46 2022] PUT /identity/v3/projects/96a4c885eada487d849017e4e6e981eb/users/255694a8bcd041eabb35733448eed42d/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:46.293195 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda349c6-c677-48cb-b3fa-b7d67114d64e None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:46.293486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda349c6-c677-48cb-b3fa-b7d67114d64e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:46.293801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda349c6-c677-48cb-b3fa-b7d67114d64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:46.326208 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:46.337043 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-cda349c6-c677-48cb-b3fa-b7d67114d64e None None] MFA Rules not processed for user `255694a8bcd041eabb35733448eed42d`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:46.338227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cda349c6-c677-48cb-b3fa-b7d67114d64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:46.350282 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb41a5-090d-48f4-9ef6-f79f23b3ea9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:46.352869 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb41a5-090d-48f4-9ef6-f79f23b3ea9a service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:46.366797 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1023/2064] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:46 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:02:46.414840 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1042/2065] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:02:46.429122 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb41a5-090d-48f4-9ef6-f79f23b3ea9a service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:46.432501 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e5bb41a5-090d-48f4-9ef6-f79f23b3ea9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:46.526689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda349c6-c677-48cb-b3fa-b7d67114d64e None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:46.526852 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb41a5-090d-48f4-9ef6-f79f23b3ea9a service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:46.528429 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cda349c6-c677-48cb-b3fa-b7d67114d64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:46.603567 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda349c6-c677-48cb-b3fa-b7d67114d64e None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:46.605494 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc80b0a-8bfd-47c6-ac9b-de2d5e848bee service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb18199dc8b42589184042556e98983', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:46.606145 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc80b0a-8bfd-47c6-ac9b-de2d5e848bee service glance] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:46.606429 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc80b0a-8bfd-47c6-ac9b-de2d5e848bee service glance] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:46.606729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc80b0a-8bfd-47c6-ac9b-de2d5e848bee service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:46.608516 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258b7b9-2f5d-4a8a-bbcf-fc76391bb212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:46.609192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258b7b9-2f5d-4a8a-bbcf-fc76391bb212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:46.609471 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258b7b9-2f5d-4a8a-bbcf-fc76391bb212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:46.609744 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258b7b9-2f5d-4a8a-bbcf-fc76391bb212 admin admin] PATH_INFO: `/v3/projects/31556c31a81a4895bed0e6493b32bc71` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:46.610892 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258b7b9-2f5d-4a8a-bbcf-fc76391bb212 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31556c31a81a4895bed0e6493b32bc71)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:46.613623 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258b7b9-2f5d-4a8a-bbcf-fc76391bb212 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:46.616212 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc80b0a-8bfd-47c6-ac9b-de2d5e848bee service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:46.617589 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc80b0a-8bfd-47c6-ac9b-de2d5e848bee service glance] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:46.634649 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1024/2066] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:02:46.635929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc80b0a-8bfd-47c6-ac9b-de2d5e848bee service glance] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:46.696573 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc80b0a-8bfd-47c6-ac9b-de2d5e848bee service glance] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:46.700284 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70162ff9-638a-490e-ae46-d9bd7e0ec8b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:46.700890 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70162ff9-638a-490e-ae46-d9bd7e0ec8b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:46.701172 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70162ff9-638a-490e-ae46-d9bd7e0ec8b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:46.701447 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70162ff9-638a-490e-ae46-d9bd7e0ec8b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:46.703222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-70162ff9-638a-490e-ae46-d9bd7e0ec8b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:46.726427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1258b7b9-2f5d-4a8a-bbcf-fc76391bb212 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31556c31a81a4895bed0e6493b32bc71', 'request_id': 'req-1258b7b9-2f5d-4a8a-bbcf-fc76391bb212'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:46.768027 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70162ff9-638a-490e-ae46-d9bd7e0ec8b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:46.769556 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70162ff9-638a-490e-ae46-d9bd7e0ec8b7 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:46.779159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1025/2067] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:02:46.792804 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1258b7b9-2f5d-4a8a-bbcf-fc76391bb212 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31556c31a81a4895bed0e6493b32bc71', 'request_id': 'req-1258b7b9-2f5d-4a8a-bbcf-fc76391bb212'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:46.800482 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1258b7b9-2f5d-4a8a-bbcf-fc76391bb212 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31556c31a81a4895bed0e6493b32bc71', 'request_id': 'req-1258b7b9-2f5d-4a8a-bbcf-fc76391bb212'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:46.813898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1258b7b9-2f5d-4a8a-bbcf-fc76391bb212 admin admin] The token cache is being invalidate because project 31556c31a81a4895bed0e6493b32bc71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:46.814215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1258b7b9-2f5d-4a8a-bbcf-fc76391bb212 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:46.814935 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1258b7b9-2f5d-4a8a-bbcf-fc76391bb212 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:46.816514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1043/2068] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:02:46 2022] DELETE /identity/v3/projects/31556c31a81a4895bed0e6493b32bc71 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:46.821698 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7cf6c-19cd-4412-a698-65f7f829ff56 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:46.821983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7cf6c-19cd-4412-a698-65f7f829ff56 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:46.822256 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7cf6c-19cd-4412-a698-65f7f829ff56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:46.846491 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-24b7cf6c-19cd-4412-a698-65f7f829ff56 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:46.848103 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-24b7cf6c-19cd-4412-a698-65f7f829ff56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:46.912336 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1026/2069] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:46.920368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed936ce6-05ec-4791-8ea1-a516d4616a0c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:46.920699 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed936ce6-05ec-4791-8ea1-a516d4616a0c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:46.920980 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed936ce6-05ec-4791-8ea1-a516d4616a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:46.943360 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-ed936ce6-05ec-4791-8ea1-a516d4616a0c None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:46.944580 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ed936ce6-05ec-4791-8ea1-a516d4616a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:46.953244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1044/2070] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:46.958209 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7cf6c-19cd-4412-a698-65f7f829ff56 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:46.968785 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7cf6c-19cd-4412-a698-65f7f829ff56 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:46.976705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db28c55-9885-4099-b57b-453b6275c996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:46.977325 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db28c55-9885-4099-b57b-453b6275c996 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:46.977717 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db28c55-9885-4099-b57b-453b6275c996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:46.977891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db28c55-9885-4099-b57b-453b6275c996 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:46.978993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db28c55-9885-4099-b57b-453b6275c996 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:02:46.979338 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db28c55-9885-4099-b57b-453b6275c996 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:46.981350 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db28c55-9885-4099-b57b-453b6275c996 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:46.994374 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1027/2071] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:02:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:02:46.998739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed936ce6-05ec-4791-8ea1-a516d4616a0c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:47.008779 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed936ce6-05ec-4791-8ea1-a516d4616a0c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:47.013562 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4fa02-d3e5-42ca-8113-404d7b7947da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:47.014155 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4fa02-d3e5-42ca-8113-404d7b7947da admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:47.014427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4fa02-d3e5-42ca-8113-404d7b7947da admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:47.014718 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4fa02-d3e5-42ca-8113-404d7b7947da admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:47.015384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4fa02-d3e5-42ca-8113-404d7b7947da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:47.017272 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4fa02-d3e5-42ca-8113-404d7b7947da admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:47.212544 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1045/2072] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:02:46 2022] POST /identity/v3/projects => generated 367 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:02:47.216641 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db28c55-9885-4099-b57b-453b6275c996 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:47.229451 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db28c55-9885-4099-b57b-453b6275c996 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:47.286691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023cb69-ce80-4c53-9cdc-e38f8786c23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:47.287302 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023cb69-ce80-4c53-9cdc-e38f8786c23f admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:47.287573 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023cb69-ce80-4c53-9cdc-e38f8786c23f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:47.287841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023cb69-ce80-4c53-9cdc-e38f8786c23f admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:47.288464 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023cb69-ce80-4c53-9cdc-e38f8786c23f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:47.290376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023cb69-ce80-4c53-9cdc-e38f8786c23f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:47.375448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1028/2073] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:02:47 2022] POST /identity/v3/users => generated 284 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:47.379790 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4fa02-d3e5-42ca-8113-404d7b7947da admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:47.389841 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4fa02-d3e5-42ca-8113-404d7b7947da admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:47.395613 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ef887-d87b-44f0-9b81-b266e9fd4c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:47.396212 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ef887-d87b-44f0-9b81-b266e9fd4c18 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:47.396488 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ef887-d87b-44f0-9b81-b266e9fd4c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:47.396754 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ef887-d87b-44f0-9b81-b266e9fd4c18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:47.398212 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580ef887-d87b-44f0-9b81-b266e9fd4c18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:47.399138 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580ef887-d87b-44f0-9b81-b266e9fd4c18 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:47.413234 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1046/2074] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:47.413429 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:47.415467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023cb69-ce80-4c53-9cdc-e38f8786c23f admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:47.428030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023cb69-ce80-4c53-9cdc-e38f8786c23f admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:47.433153 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f4064-fbb9-4fae-a648-0ea054388edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:47.434157 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f4064-fbb9-4fae-a648-0ea054388edb admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:47.434453 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f4064-fbb9-4fae-a648-0ea054388edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:47.434941 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f4064-fbb9-4fae-a648-0ea054388edb admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:47.435701 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f4064-fbb9-4fae-a648-0ea054388edb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:47.442832 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f4064-fbb9-4fae-a648-0ea054388edb admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:47.452536 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1029/2075] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:47.457376 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ef887-d87b-44f0-9b81-b266e9fd4c18 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:47.464244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:47.469217 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ef887-d87b-44f0-9b81-b266e9fd4c18 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:47.474498 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441a821-09ce-41ea-b3a4-604a1600745e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:47.475212 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441a821-09ce-41ea-b3a4-604a1600745e admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:47.475488 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441a821-09ce-41ea-b3a4-604a1600745e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:47.475756 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441a821-09ce-41ea-b3a4-604a1600745e admin admin] PATH_INFO: `/v3/projects/797bdf2cd3fe48e9981b82043de3b919/users/9d7c832f5b894850b508d11916410eee/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:47.489931 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0441a821-09ce-41ea-b3a4-604a1600745e admin admin] RBAC: Authorizing `identity:create_grant(project_id=797bdf2cd3fe48e9981b82043de3b919, user_id=9d7c832f5b894850b508d11916410eee, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:47.492602 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0441a821-09ce-41ea-b3a4-604a1600745e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:47.609707 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1047/2076] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:47 2022] PUT /identity/v3/projects/797bdf2cd3fe48e9981b82043de3b919/users/9d7c832f5b894850b508d11916410eee/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:47.622436 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031b277-9604-40d5-84f1-0861f6e67974 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:47.625335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031b277-9604-40d5-84f1-0861f6e67974 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:47.625335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031b277-9604-40d5-84f1-0861f6e67974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:47.663659 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d031b277-9604-40d5-84f1-0861f6e67974 None None] MFA Rules not processed for user `9d7c832f5b894850b508d11916410eee`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:47.665156 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d031b277-9604-40d5-84f1-0861f6e67974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:47.754096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1030/2077] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:47.759139 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441a821-09ce-41ea-b3a4-604a1600745e admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:47.843679 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441a821-09ce-41ea-b3a4-604a1600745e admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:47.849816 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7888fa2-fae1-4075-af5c-a98d8fc92ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:47.850418 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7888fa2-fae1-4075-af5c-a98d8fc92ddb admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:47.850728 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7888fa2-fae1-4075-af5c-a98d8fc92ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:47.851002 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7888fa2-fae1-4075-af5c-a98d8fc92ddb admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:47.851642 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7888fa2-fae1-4075-af5c-a98d8fc92ddb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:47.853769 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7888fa2-fae1-4075-af5c-a98d8fc92ddb admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:47.944552 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1048/2078] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:02:47 2022] POST /identity/v3/projects => generated 369 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:02:47.950767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031b277-9604-40d5-84f1-0861f6e67974 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:47.962191 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031b277-9604-40d5-84f1-0861f6e67974 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:48.016562 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:48.026361 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7a2226-caf1-499c-a2de-01355d21009b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:48.027007 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a2226-caf1-499c-a2de-01355d21009b admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:48.027287 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a2226-caf1-499c-a2de-01355d21009b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:48.027631 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a2226-caf1-499c-a2de-01355d21009b admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:48.028212 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7a2226-caf1-499c-a2de-01355d21009b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:48.030165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7a2226-caf1-499c-a2de-01355d21009b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:48.419591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1031/2079] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:02:47 2022] POST /identity/v3/users => generated 291 bytes in 468 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:48.421562 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7888fa2-fae1-4075-af5c-a98d8fc92ddb admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:48.435559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7888fa2-fae1-4075-af5c-a98d8fc92ddb admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:48.439063 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3353ed-d416-4d81-ad56-b0debba48459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:48.439690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3353ed-d416-4d81-ad56-b0debba48459 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:48.439963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3353ed-d416-4d81-ad56-b0debba48459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:48.440227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3353ed-d416-4d81-ad56-b0debba48459 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:48.440860 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3353ed-d416-4d81-ad56-b0debba48459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:48.442575 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3353ed-d416-4d81-ad56-b0debba48459 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:48.453770 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1049/2080] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:02:48.458732 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7a2226-caf1-499c-a2de-01355d21009b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:48.468613 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7a2226-caf1-499c-a2de-01355d21009b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:48.474433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06e837-bfcd-4ec5-ad8d-8740955a7ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:48.475158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06e837-bfcd-4ec5-ad8d-8740955a7ca4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:48.481597 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06e837-bfcd-4ec5-ad8d-8740955a7ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:48.481597 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06e837-bfcd-4ec5-ad8d-8740955a7ca4 admin admin] PATH_INFO: `/v3/projects/5c6a3b9062aa40048ea282ec9408243c/users/1c04164cfdc646988e6649590a1658ff/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:48.493423 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06e837-bfcd-4ec5-ad8d-8740955a7ca4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c6a3b9062aa40048ea282ec9408243c, user_id=1c04164cfdc646988e6649590a1658ff, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:48.496421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06e837-bfcd-4ec5-ad8d-8740955a7ca4 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:48.599471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1032/2081] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:48 2022] PUT /identity/v3/projects/5c6a3b9062aa40048ea282ec9408243c/users/1c04164cfdc646988e6649590a1658ff/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:48.603652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3353ed-d416-4d81-ad56-b0debba48459 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:48.613983 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3353ed-d416-4d81-ad56-b0debba48459 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:48.680800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470f065-a234-4937-a035-c2a5fafc1245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:48.681609 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470f065-a234-4937-a035-c2a5fafc1245 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:48.681974 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470f065-a234-4937-a035-c2a5fafc1245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:48.682339 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470f065-a234-4937-a035-c2a5fafc1245 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:48.683389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1470f065-a234-4937-a035-c2a5fafc1245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:48.685160 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1470f065-a234-4937-a035-c2a5fafc1245 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:48.698181 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1050/2082] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:48.703831 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06e837-bfcd-4ec5-ad8d-8740955a7ca4 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:48.729177 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06e837-bfcd-4ec5-ad8d-8740955a7ca4 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:48.734490 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023935a-6c36-48cf-a10b-8cb8ad1c46fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:48.735192 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023935a-6c36-48cf-a10b-8cb8ad1c46fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:48.735466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023935a-6c36-48cf-a10b-8cb8ad1c46fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:48.735740 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023935a-6c36-48cf-a10b-8cb8ad1c46fb admin admin] PATH_INFO: `/v3/projects/5c6a3b9062aa40048ea282ec9408243c/users/1c04164cfdc646988e6649590a1658ff/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:48.738931 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6023935a-6c36-48cf-a10b-8cb8ad1c46fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c6a3b9062aa40048ea282ec9408243c, user_id=1c04164cfdc646988e6649590a1658ff, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:48.741809 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6023935a-6c36-48cf-a10b-8cb8ad1c46fb admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:48.754578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1033/2083] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:48 2022] PUT /identity/v3/projects/5c6a3b9062aa40048ea282ec9408243c/users/1c04164cfdc646988e6649590a1658ff/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:48.764789 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ab999-3792-4803-bc6f-6d4b403d00db None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:48.765082 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ab999-3792-4803-bc6f-6d4b403d00db None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:48.765359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ab999-3792-4803-bc6f-6d4b403d00db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:48.770217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:48.805639 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-c83ab999-3792-4803-bc6f-6d4b403d00db None None] MFA Rules not processed for user `1c04164cfdc646988e6649590a1658ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:48.806902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c83ab999-3792-4803-bc6f-6d4b403d00db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:48.881104 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:48.904721 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1051/2084] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:48.961853 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023935a-6c36-48cf-a10b-8cb8ad1c46fb admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:48.963584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6023935a-6c36-48cf-a10b-8cb8ad1c46fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:49.035773 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023935a-6c36-48cf-a10b-8cb8ad1c46fb admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:49.040113 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c032e-a770-4f8c-8966-ccad7e5f5a7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:49.040726 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c032e-a770-4f8c-8966-ccad7e5f5a7b service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:49.041008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c032e-a770-4f8c-8966-ccad7e5f5a7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:49.041280 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c032e-a770-4f8c-8966-ccad7e5f5a7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:49.051445 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c032e-a770-4f8c-8966-ccad7e5f5a7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:49.052890 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c032e-a770-4f8c-8966-ccad7e5f5a7b service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:49.062431 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1034/2085] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:48 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:02:49.096481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:52.279910 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ab999-3792-4803-bc6f-6d4b403d00db None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:52.350385 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ab999-3792-4803-bc6f-6d4b403d00db None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:52.355138 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88307455-48e8-4181-9284-c2c2112463a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:52.355762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88307455-48e8-4181-9284-c2c2112463a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:52.356035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88307455-48e8-4181-9284-c2c2112463a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:52.356299 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88307455-48e8-4181-9284-c2c2112463a9 admin admin] PATH_INFO: `/v3/users/9d7c832f5b894850b508d11916410eee` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:52.357421 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88307455-48e8-4181-9284-c2c2112463a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d7c832f5b894850b508d11916410eee)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:52.359571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88307455-48e8-4181-9284-c2c2112463a9 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:52.599550 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-88307455-48e8-4181-9284-c2c2112463a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d7c832f5b894850b508d11916410eee'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:52.612001 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-88307455-48e8-4181-9284-c2c2112463a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d7c832f5b894850b508d11916410eee'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:52.619062 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-88307455-48e8-4181-9284-c2c2112463a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d7c832f5b894850b508d11916410eee'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:52.619761 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-88307455-48e8-4181-9284-c2c2112463a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d7c832f5b894850b508d11916410eee'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:52.621459 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:52.767812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-88307455-48e8-4181-9284-c2c2112463a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d7c832f5b894850b508d11916410eee'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:52.776131 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1052/2086] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:02:52 2022] DELETE /identity/v3/users/9d7c832f5b894850b508d11916410eee => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:52.777480 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7d7f8-897b-40da-8758-9827c329db6a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:52.778040 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7d7f8-897b-40da-8758-9827c329db6a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:52.778040 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7d7f8-897b-40da-8758-9827c329db6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:52.798473 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-72e7d7f8-897b-40da-8758-9827c329db6a None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:52.801010 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-72e7d7f8-897b-40da-8758-9827c329db6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:52.873777 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1035/2087] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:52.886071 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88307455-48e8-4181-9284-c2c2112463a9 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:52.887905 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-88307455-48e8-4181-9284-c2c2112463a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:52.950581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88307455-48e8-4181-9284-c2c2112463a9 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:52.954406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6a52b-b3ab-42d4-8ad7-1c9009cafad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:52.955649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6a52b-b3ab-42d4-8ad7-1c9009cafad6 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:52.957410 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6a52b-b3ab-42d4-8ad7-1c9009cafad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:52.957410 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6a52b-b3ab-42d4-8ad7-1c9009cafad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:52.966141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6a52b-b3ab-42d4-8ad7-1c9009cafad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:52.967795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6a52b-b3ab-42d4-8ad7-1c9009cafad6 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:52.979503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1053/2088] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:02:53.435876 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7d7f8-897b-40da-8758-9827c329db6a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:53.438228 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-72e7d7f8-897b-40da-8758-9827c329db6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:53.460670 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7d7f8-897b-40da-8758-9827c329db6a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:53.485662 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5627d6-d65a-47c4-8e5e-cd36d630801b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:53.487025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5627d6-d65a-47c4-8e5e-cd36d630801b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:53.487227 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5627d6-d65a-47c4-8e5e-cd36d630801b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:53.487457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5627d6-d65a-47c4-8e5e-cd36d630801b admin admin] PATH_INFO: `/v3/projects/797bdf2cd3fe48e9981b82043de3b919` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:53.488666 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5627d6-d65a-47c4-8e5e-cd36d630801b admin admin] RBAC: Authorizing `identity:delete_project(project_id=797bdf2cd3fe48e9981b82043de3b919)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:53.490589 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5627d6-d65a-47c4-8e5e-cd36d630801b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:53.667969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9b5627d6-d65a-47c4-8e5e-cd36d630801b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '797bdf2cd3fe48e9981b82043de3b919', 'request_id': 'req-9b5627d6-d65a-47c4-8e5e-cd36d630801b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:53.684086 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9b5627d6-d65a-47c4-8e5e-cd36d630801b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '797bdf2cd3fe48e9981b82043de3b919', 'request_id': 'req-9b5627d6-d65a-47c4-8e5e-cd36d630801b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:53.700697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9b5627d6-d65a-47c4-8e5e-cd36d630801b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '797bdf2cd3fe48e9981b82043de3b919', 'request_id': 'req-9b5627d6-d65a-47c4-8e5e-cd36d630801b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:53.796787 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9b5627d6-d65a-47c4-8e5e-cd36d630801b admin admin] The token cache is being invalidate because project 797bdf2cd3fe48e9981b82043de3b919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:53.797958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9b5627d6-d65a-47c4-8e5e-cd36d630801b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:53.799522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9b5627d6-d65a-47c4-8e5e-cd36d630801b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:53.801545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1036/2089] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:02:53 2022] DELETE /identity/v3/projects/797bdf2cd3fe48e9981b82043de3b919 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:53.805867 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6a52b-b3ab-42d4-8ad7-1c9009cafad6 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:53.818906 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6a52b-b3ab-42d4-8ad7-1c9009cafad6 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:53.906600 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e62d9e-e731-4812-a4c1-22565be047a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:53.907912 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e62d9e-e731-4812-a4c1-22565be047a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:53.908786 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e62d9e-e731-4812-a4c1-22565be047a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:53.909059 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e62d9e-e731-4812-a4c1-22565be047a4 admin admin] PATH_INFO: `/v3/users/1c04164cfdc646988e6649590a1658ff` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:53.910221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e62d9e-e731-4812-a4c1-22565be047a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c04164cfdc646988e6649590a1658ff)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:53.912227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e62d9e-e731-4812-a4c1-22565be047a4 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:54.161519 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d4e62d9e-e731-4812-a4c1-22565be047a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c04164cfdc646988e6649590a1658ff'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:54.175713 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d4e62d9e-e731-4812-a4c1-22565be047a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c04164cfdc646988e6649590a1658ff'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:54.183722 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d4e62d9e-e731-4812-a4c1-22565be047a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c04164cfdc646988e6649590a1658ff'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:54.184552 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d4e62d9e-e731-4812-a4c1-22565be047a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c04164cfdc646988e6649590a1658ff'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:54.244416 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d4e62d9e-e731-4812-a4c1-22565be047a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c04164cfdc646988e6649590a1658ff'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:54.251102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1054/2090] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:02:53 2022] DELETE /identity/v3/users/1c04164cfdc646988e6649590a1658ff => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:54.733724 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5627d6-d65a-47c4-8e5e-cd36d630801b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:54.735280 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9b5627d6-d65a-47c4-8e5e-cd36d630801b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:54.750060 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5627d6-d65a-47c4-8e5e-cd36d630801b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:54.791172 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:54.860143 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0077c02-1884-45ec-842b-bcdb0e91e5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:54.860843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0077c02-1884-45ec-842b-bcdb0e91e5d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:54.861126 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0077c02-1884-45ec-842b-bcdb0e91e5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:54.861928 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0077c02-1884-45ec-842b-bcdb0e91e5d8 admin admin] PATH_INFO: `/v3/projects/5c6a3b9062aa40048ea282ec9408243c` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:54.863777 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0077c02-1884-45ec-842b-bcdb0e91e5d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c6a3b9062aa40048ea282ec9408243c)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:54.866337 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0077c02-1884-45ec-842b-bcdb0e91e5d8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:54.985133 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b0077c02-1884-45ec-842b-bcdb0e91e5d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c6a3b9062aa40048ea282ec9408243c', 'request_id': 'req-b0077c02-1884-45ec-842b-bcdb0e91e5d8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:54.992180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b0077c02-1884-45ec-842b-bcdb0e91e5d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c6a3b9062aa40048ea282ec9408243c', 'request_id': 'req-b0077c02-1884-45ec-842b-bcdb0e91e5d8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:54.998002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b0077c02-1884-45ec-842b-bcdb0e91e5d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c6a3b9062aa40048ea282ec9408243c', 'request_id': 'req-b0077c02-1884-45ec-842b-bcdb0e91e5d8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:55.085077 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b0077c02-1884-45ec-842b-bcdb0e91e5d8 admin admin] The token cache is being invalidate because project 5c6a3b9062aa40048ea282ec9408243c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:55.085553 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b0077c02-1884-45ec-842b-bcdb0e91e5d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:55.086209 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b0077c02-1884-45ec-842b-bcdb0e91e5d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:55.087998 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1037/2091] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:02:54 2022] DELETE /identity/v3/projects/5c6a3b9062aa40048ea282ec9408243c => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:55.093917 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6638d-1def-4d48-b619-8fd80c93902c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:55.094424 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6638d-1def-4d48-b619-8fd80c93902c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:55.094536 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6638d-1def-4d48-b619-8fd80c93902c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:55.117370 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-eed6638d-1def-4d48-b619-8fd80c93902c None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:55.119300 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-eed6638d-1def-4d48-b619-8fd80c93902c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:55.132416 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:55.202986 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1055/2092] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:55.213086 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0d764-14e5-478d-bea2-013b707a8ad3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:55.213448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0d764-14e5-478d-bea2-013b707a8ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:55.213806 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0d764-14e5-478d-bea2-013b707a8ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:55.234904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-47f0d764-14e5-478d-bea2-013b707a8ad3 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:55.240386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-47f0d764-14e5-478d-bea2-013b707a8ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:55.244256 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1038/2093] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:55.248658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6638d-1def-4d48-b619-8fd80c93902c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:55.259833 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6638d-1def-4d48-b619-8fd80c93902c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:55.264583 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c1a87-f29d-4587-b3a6-799ada40150b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:55.269163 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c1a87-f29d-4587-b3a6-799ada40150b admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:55.269163 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c1a87-f29d-4587-b3a6-799ada40150b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:55.269163 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c1a87-f29d-4587-b3a6-799ada40150b admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:55.269163 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c1a87-f29d-4587-b3a6-799ada40150b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:02:55.269163 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c1a87-f29d-4587-b3a6-799ada40150b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:55.269163 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c1a87-f29d-4587-b3a6-799ada40150b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:55.281581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1056/2094] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:02:55.286306 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0d764-14e5-478d-bea2-013b707a8ad3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:55.295871 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0d764-14e5-478d-bea2-013b707a8ad3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:55.301079 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea7e34-8566-4631-ac22-39b58d754bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:55.301874 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea7e34-8566-4631-ac22-39b58d754bfd admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:55.302151 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea7e34-8566-4631-ac22-39b58d754bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:55.302420 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea7e34-8566-4631-ac22-39b58d754bfd admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:55.303132 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea7e34-8566-4631-ac22-39b58d754bfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:55.305193 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea7e34-8566-4631-ac22-39b58d754bfd admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:55.416594 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1039/2095] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:02:55 2022] POST /identity/v3/projects => generated 371 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:55.421568 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c1a87-f29d-4587-b3a6-799ada40150b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:55.434112 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c1a87-f29d-4587-b3a6-799ada40150b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:55.494808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96270df-c4f9-4f0f-b7d2-d6da113a42a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:55.495342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96270df-c4f9-4f0f-b7d2-d6da113a42a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:55.495621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96270df-c4f9-4f0f-b7d2-d6da113a42a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:55.495896 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96270df-c4f9-4f0f-b7d2-d6da113a42a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:55.496543 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96270df-c4f9-4f0f-b7d2-d6da113a42a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:55.498473 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96270df-c4f9-4f0f-b7d2-d6da113a42a0 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:55.591839 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1057/2096] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:02:55 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:55.596031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea7e34-8566-4631-ac22-39b58d754bfd admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:55.606357 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea7e34-8566-4631-ac22-39b58d754bfd admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:55.613229 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2853a119-e6b3-40d1-894e-b4f26a4b436b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:55.613828 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2853a119-e6b3-40d1-894e-b4f26a4b436b admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:55.614149 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2853a119-e6b3-40d1-894e-b4f26a4b436b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:55.614420 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2853a119-e6b3-40d1-894e-b4f26a4b436b admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:55.615261 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2853a119-e6b3-40d1-894e-b4f26a4b436b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:55.616946 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2853a119-e6b3-40d1-894e-b4f26a4b436b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:55.627668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1040/2097] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:55.633688 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96270df-c4f9-4f0f-b7d2-d6da113a42a0 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:55.643737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96270df-c4f9-4f0f-b7d2-d6da113a42a0 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:55.648401 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089fa28-d395-42ad-9e79-9795e1896456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:55.648984 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089fa28-d395-42ad-9e79-9795e1896456 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:55.649262 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089fa28-d395-42ad-9e79-9795e1896456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:55.649534 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089fa28-d395-42ad-9e79-9795e1896456 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:55.650166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089fa28-d395-42ad-9e79-9795e1896456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:55.651793 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089fa28-d395-42ad-9e79-9795e1896456 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:55.662928 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1058/2098] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:55.667068 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2853a119-e6b3-40d1-894e-b4f26a4b436b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:55.677774 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2853a119-e6b3-40d1-894e-b4f26a4b436b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:55.682323 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7e0761-d55f-4eb0-9150-b2a2b940fa4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:55.683022 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e0761-d55f-4eb0-9150-b2a2b940fa4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:55.683298 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e0761-d55f-4eb0-9150-b2a2b940fa4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:55.683570 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e0761-d55f-4eb0-9150-b2a2b940fa4c admin admin] PATH_INFO: `/v3/projects/067c8ff4ac7f4a3399d854f862098a4c/users/334a9483a355427c8b7e4dfe37a6f7b2/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:55.699106 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7e0761-d55f-4eb0-9150-b2a2b940fa4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=067c8ff4ac7f4a3399d854f862098a4c, user_id=334a9483a355427c8b7e4dfe37a6f7b2, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:55.701787 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7e0761-d55f-4eb0-9150-b2a2b940fa4c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:55.781624 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1041/2099] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:55 2022] PUT /identity/v3/projects/067c8ff4ac7f4a3399d854f862098a4c/users/334a9483a355427c8b7e4dfe37a6f7b2/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:55.792079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af3017-e864-4b7a-a47b-88e8ee3e6aca None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:55.792359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af3017-e864-4b7a-a47b-88e8ee3e6aca None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:55.792635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af3017-e864-4b7a-a47b-88e8ee3e6aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:55.824354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-63af3017-e864-4b7a-a47b-88e8ee3e6aca None None] MFA Rules not processed for user `334a9483a355427c8b7e4dfe37a6f7b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:55.825662 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-63af3017-e864-4b7a-a47b-88e8ee3e6aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:55.891274 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1059/2100] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:55.906039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7e0761-d55f-4eb0-9150-b2a2b940fa4c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:55.907605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-bc7e0761-d55f-4eb0-9150-b2a2b940fa4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:55.967813 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7e0761-d55f-4eb0-9150-b2a2b940fa4c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:55.972465 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca362a2-acc0-47d7-bf27-c08b03ae8ba8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:55.973094 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca362a2-acc0-47d7-bf27-c08b03ae8ba8 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:55.973394 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca362a2-acc0-47d7-bf27-c08b03ae8ba8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:55.973649 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca362a2-acc0-47d7-bf27-c08b03ae8ba8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:55.979955 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:55.982286 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca362a2-acc0-47d7-bf27-c08b03ae8ba8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:55.983882 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca362a2-acc0-47d7-bf27-c08b03ae8ba8 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:55.994012 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1042/2101] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:02:56.026458 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af3017-e864-4b7a-a47b-88e8ee3e6aca None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:56.037536 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:56.098260 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af3017-e864-4b7a-a47b-88e8ee3e6aca None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:56.103388 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665f127-3b3b-4893-a39f-1ab0019a2434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:56.104001 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665f127-3b3b-4893-a39f-1ab0019a2434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:56.104281 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665f127-3b3b-4893-a39f-1ab0019a2434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:56.104555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665f127-3b3b-4893-a39f-1ab0019a2434 admin admin] PATH_INFO: `/v3/users/334a9483a355427c8b7e4dfe37a6f7b2` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:56.105781 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665f127-3b3b-4893-a39f-1ab0019a2434 admin admin] RBAC: Authorizing `identity:delete_user(user_id=334a9483a355427c8b7e4dfe37a6f7b2)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:56.108062 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665f127-3b3b-4893-a39f-1ab0019a2434 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:56.212189 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca362a2-acc0-47d7-bf27-c08b03ae8ba8 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:56.213731 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2ca362a2-acc0-47d7-bf27-c08b03ae8ba8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:56.228701 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca362a2-acc0-47d7-bf27-c08b03ae8ba8 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:56.233278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf8c31-ec59-42c0-9f74-f035e51ec4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:56.233922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf8c31-ec59-42c0-9f74-f035e51ec4d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:56.234196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf8c31-ec59-42c0-9f74-f035e51ec4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:56.234573 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf8c31-ec59-42c0-9f74-f035e51ec4d5 admin admin] PATH_INFO: `/v3/users/9de6519722ba4804943751da63d1b45a` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:56.235660 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf8c31-ec59-42c0-9f74-f035e51ec4d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9de6519722ba4804943751da63d1b45a)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:56.238203 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf8c31-ec59-42c0-9f74-f035e51ec4d5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:56.358919 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2665f127-3b3b-4893-a39f-1ab0019a2434 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '334a9483a355427c8b7e4dfe37a6f7b2'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:56.372325 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2665f127-3b3b-4893-a39f-1ab0019a2434 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '334a9483a355427c8b7e4dfe37a6f7b2'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:56.380183 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2665f127-3b3b-4893-a39f-1ab0019a2434 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '334a9483a355427c8b7e4dfe37a6f7b2'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:56.380923 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2665f127-3b3b-4893-a39f-1ab0019a2434 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '334a9483a355427c8b7e4dfe37a6f7b2'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:56.432141 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4baf8c31-ec59-42c0-9f74-f035e51ec4d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9de6519722ba4804943751da63d1b45a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:56.440481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4baf8c31-ec59-42c0-9f74-f035e51ec4d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9de6519722ba4804943751da63d1b45a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:56.463350 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4baf8c31-ec59-42c0-9f74-f035e51ec4d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9de6519722ba4804943751da63d1b45a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:56.465300 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2665f127-3b3b-4893-a39f-1ab0019a2434 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '334a9483a355427c8b7e4dfe37a6f7b2'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:56.472433 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1060/2102] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:02:56 2022] DELETE /identity/v3/users/334a9483a355427c8b7e4dfe37a6f7b2 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:56.476943 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2de913-e99b-46c6-88f4-1ec558141296 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:56.477252 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2de913-e99b-46c6-88f4-1ec558141296 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:56.477526 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2de913-e99b-46c6-88f4-1ec558141296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:56.497375 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-bb2de913-e99b-46c6-88f4-1ec558141296 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:56.498569 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-bb2de913-e99b-46c6-88f4-1ec558141296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:56.536284 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4baf8c31-ec59-42c0-9f74-f035e51ec4d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9de6519722ba4804943751da63d1b45a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:56.536284 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4baf8c31-ec59-42c0-9f74-f035e51ec4d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9de6519722ba4804943751da63d1b45a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:56.547236 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1043/2103] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:02:56 2022] DELETE /identity/v3/users/9de6519722ba4804943751da63d1b45a => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:56.549396 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febb16c-50cd-4bd2-ba52-e6e9a8836fc6 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:56.549730 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febb16c-50cd-4bd2-ba52-e6e9a8836fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:56.550013 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febb16c-50cd-4bd2-ba52-e6e9a8836fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:56.574756 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-2febb16c-50cd-4bd2-ba52-e6e9a8836fc6 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:56.575898 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2febb16c-50cd-4bd2-ba52-e6e9a8836fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:56.581057 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1061/2104] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:02:56.588167 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2de913-e99b-46c6-88f4-1ec558141296 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:56.588167 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-bb2de913-e99b-46c6-88f4-1ec558141296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:56.596423 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2de913-e99b-46c6-88f4-1ec558141296 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:56.625651 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:56.643557 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1044/2105] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:56.644521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febb16c-50cd-4bd2-ba52-e6e9a8836fc6 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:56.645795 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2febb16c-50cd-4bd2-ba52-e6e9a8836fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:56.663012 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f091bfa-44c9-40e0-8c59-a93e04959562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:56.663696 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f091bfa-44c9-40e0-8c59-a93e04959562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:56.663959 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f091bfa-44c9-40e0-8c59-a93e04959562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:56.664227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f091bfa-44c9-40e0-8c59-a93e04959562 admin admin] PATH_INFO: `/v3/users/dc4f11207b6046ffb56ccdb4e63103b5` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:56.665366 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f091bfa-44c9-40e0-8c59-a93e04959562 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc4f11207b6046ffb56ccdb4e63103b5)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:56.667339 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f091bfa-44c9-40e0-8c59-a93e04959562 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:56.708612 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febb16c-50cd-4bd2-ba52-e6e9a8836fc6 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:56.712331 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a754df-1a27-4d55-a7be-fee2d9014f90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:56.712959 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a754df-1a27-4d55-a7be-fee2d9014f90 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:56.713236 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a754df-1a27-4d55-a7be-fee2d9014f90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:56.713505 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a754df-1a27-4d55-a7be-fee2d9014f90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:56.722330 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a754df-1a27-4d55-a7be-fee2d9014f90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:56.724386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a754df-1a27-4d55-a7be-fee2d9014f90 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:56.734142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1045/2106] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:02:56.735087 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a754df-1a27-4d55-a7be-fee2d9014f90 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:56.736424 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b7a754df-1a27-4d55-a7be-fee2d9014f90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:56.748804 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a754df-1a27-4d55-a7be-fee2d9014f90 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:56.752217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f3d96-4cd9-4b29-8523-7ec5083e5895 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:56.752807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f3d96-4cd9-4b29-8523-7ec5083e5895 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:56.753081 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f3d96-4cd9-4b29-8523-7ec5083e5895 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:56.753366 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f3d96-4cd9-4b29-8523-7ec5083e5895 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:56.763209 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67f3d96-4cd9-4b29-8523-7ec5083e5895 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:56.764750 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67f3d96-4cd9-4b29-8523-7ec5083e5895 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:56.769087 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:56.776975 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1046/2107] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:02:56.910629 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1f091bfa-44c9-40e0-8c59-a93e04959562 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc4f11207b6046ffb56ccdb4e63103b5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:56.923150 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1f091bfa-44c9-40e0-8c59-a93e04959562 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc4f11207b6046ffb56ccdb4e63103b5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:56.931333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1f091bfa-44c9-40e0-8c59-a93e04959562 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc4f11207b6046ffb56ccdb4e63103b5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:56.932038 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1f091bfa-44c9-40e0-8c59-a93e04959562 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc4f11207b6046ffb56ccdb4e63103b5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:56.998834 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1f091bfa-44c9-40e0-8c59-a93e04959562 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc4f11207b6046ffb56ccdb4e63103b5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:57.005942 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1062/2108] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:02:56 2022] DELETE /identity/v3/users/dc4f11207b6046ffb56ccdb4e63103b5 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:57.010295 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410789cb-1e30-4652-b749-1ccff6476d55 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:57.010803 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410789cb-1e30-4652-b749-1ccff6476d55 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:57.010947 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410789cb-1e30-4652-b749-1ccff6476d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:57.032350 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-410789cb-1e30-4652-b749-1ccff6476d55 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:57.037193 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-410789cb-1e30-4652-b749-1ccff6476d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:57.103074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1047/2109] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:02:57.115990 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f091bfa-44c9-40e0-8c59-a93e04959562 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:57.117849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1f091bfa-44c9-40e0-8c59-a93e04959562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:57.193454 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f091bfa-44c9-40e0-8c59-a93e04959562 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:57.208190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884666bb-2e14-454c-b68a-578bcc492628 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:57.208847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884666bb-2e14-454c-b68a-578bcc492628 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:57.209123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884666bb-2e14-454c-b68a-578bcc492628 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:57.209397 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884666bb-2e14-454c-b68a-578bcc492628 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:57.219259 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884666bb-2e14-454c-b68a-578bcc492628 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:57.220795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884666bb-2e14-454c-b68a-578bcc492628 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:57.233372 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1063/2110] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:02:57.341710 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410789cb-1e30-4652-b749-1ccff6476d55 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:57.343516 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-410789cb-1e30-4652-b749-1ccff6476d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:57.351736 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884666bb-2e14-454c-b68a-578bcc492628 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:57.353208 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-884666bb-2e14-454c-b68a-578bcc492628 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:57.359922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410789cb-1e30-4652-b749-1ccff6476d55 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:57.371103 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c8b19-ef61-475a-8014-9254b8558c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:57.375740 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884666bb-2e14-454c-b68a-578bcc492628 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:57.375887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c8b19-ef61-475a-8014-9254b8558c7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:57.375887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c8b19-ef61-475a-8014-9254b8558c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:57.375887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c8b19-ef61-475a-8014-9254b8558c7c admin admin] PATH_INFO: `/v3/projects/067c8ff4ac7f4a3399d854f862098a4c` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:57.378056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b755d692-cd4e-448d-9747-52de97674626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:57.381689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755d692-cd4e-448d-9747-52de97674626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:57.382034 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755d692-cd4e-448d-9747-52de97674626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:57.382162 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7c8b19-ef61-475a-8014-9254b8558c7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=067c8ff4ac7f4a3399d854f862098a4c)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:57.382316 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755d692-cd4e-448d-9747-52de97674626 admin admin] PATH_INFO: `/v3/projects/a2d527452f4a4a2088368039042fa7dd` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:57.383817 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755d692-cd4e-448d-9747-52de97674626 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2d527452f4a4a2088368039042fa7dd)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:57.386498 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755d692-cd4e-448d-9747-52de97674626 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:57.386948 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7c8b19-ef61-475a-8014-9254b8558c7c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:57.551891 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:57.555374 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b755d692-cd4e-448d-9747-52de97674626 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2d527452f4a4a2088368039042fa7dd', 'request_id': 'req-b755d692-cd4e-448d-9747-52de97674626'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:57.561210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cc7c8b19-ef61-475a-8014-9254b8558c7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '067c8ff4ac7f4a3399d854f862098a4c', 'request_id': 'req-cc7c8b19-ef61-475a-8014-9254b8558c7c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:57.590332 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cc7c8b19-ef61-475a-8014-9254b8558c7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '067c8ff4ac7f4a3399d854f862098a4c', 'request_id': 'req-cc7c8b19-ef61-475a-8014-9254b8558c7c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:57.590332 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cc7c8b19-ef61-475a-8014-9254b8558c7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '067c8ff4ac7f4a3399d854f862098a4c', 'request_id': 'req-cc7c8b19-ef61-475a-8014-9254b8558c7c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:57.638023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b755d692-cd4e-448d-9747-52de97674626 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2d527452f4a4a2088368039042fa7dd', 'request_id': 'req-b755d692-cd4e-448d-9747-52de97674626'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:57.646280 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b755d692-cd4e-448d-9747-52de97674626 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2d527452f4a4a2088368039042fa7dd', 'request_id': 'req-b755d692-cd4e-448d-9747-52de97674626'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:57.661646 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b755d692-cd4e-448d-9747-52de97674626 admin admin] The token cache is being invalidate because project a2d527452f4a4a2088368039042fa7dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:57.661928 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b755d692-cd4e-448d-9747-52de97674626 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:57.662606 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b755d692-cd4e-448d-9747-52de97674626 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:57.664238 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1064/2111] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:02:57 2022] DELETE /identity/v3/projects/a2d527452f4a4a2088368039042fa7dd => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:57.669071 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d8aba-6a13-4df8-9836-d5cc67a30091 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:57.669343 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d8aba-6a13-4df8-9836-d5cc67a30091 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:57.669607 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d8aba-6a13-4df8-9836-d5cc67a30091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:57.689508 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-792d8aba-6a13-4df8-9836-d5cc67a30091 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:57.690728 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-792d8aba-6a13-4df8-9836-d5cc67a30091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:57.727654 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cc7c8b19-ef61-475a-8014-9254b8558c7c admin admin] The token cache is being invalidate because project 067c8ff4ac7f4a3399d854f862098a4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:57.727946 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cc7c8b19-ef61-475a-8014-9254b8558c7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:57.728564 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cc7c8b19-ef61-475a-8014-9254b8558c7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:57.730035 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1048/2112] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:02:57 2022] DELETE /identity/v3/projects/067c8ff4ac7f4a3399d854f862098a4c => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:57.734726 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618eb82d-977d-4f30-bfcd-d388c73cde9b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:57.735075 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618eb82d-977d-4f30-bfcd-d388c73cde9b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:57.735408 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618eb82d-977d-4f30-bfcd-d388c73cde9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:57.758811 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-618eb82d-977d-4f30-bfcd-d388c73cde9b None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:57.760360 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-618eb82d-977d-4f30-bfcd-d388c73cde9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:57.763264 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1065/2113] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:02:57.770928 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1049/2114] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:02:57.772430 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ff473-196e-4ed8-80c1-77e31da5aa87 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:57.772430 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ff473-196e-4ed8-80c1-77e31da5aa87 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:57.772430 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ff473-196e-4ed8-80c1-77e31da5aa87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:57.778445 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118ebb5-c4a8-4d89-af14-1a7f6d689c9c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:57.778768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118ebb5-c4a8-4d89-af14-1a7f6d689c9c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:57.779049 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118ebb5-c4a8-4d89-af14-1a7f6d689c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:57.792804 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-e93ff473-196e-4ed8-80c1-77e31da5aa87 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:57.795249 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e93ff473-196e-4ed8-80c1-77e31da5aa87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:57.807222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-3118ebb5-c4a8-4d89-af14-1a7f6d689c9c None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:57.808439 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3118ebb5-c4a8-4d89-af14-1a7f6d689c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:57.809360 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1066/2115] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:02:57.810395 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ff473-196e-4ed8-80c1-77e31da5aa87 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:57.811784 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e93ff473-196e-4ed8-80c1-77e31da5aa87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:57.821332 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1050/2116] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:02:57.823378 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3118ebb5-c4a8-4d89-af14-1a7f6d689c9c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:57.839832 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3118ebb5-c4a8-4d89-af14-1a7f6d689c9c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:57.841496 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ff473-196e-4ed8-80c1-77e31da5aa87 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:57.845889 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9aa6b-3f97-4404-867a-272d48618173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:57.846526 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9aa6b-3f97-4404-867a-272d48618173 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:57.846842 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9aa6b-3f97-4404-867a-272d48618173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:57.847120 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9aa6b-3f97-4404-867a-272d48618173 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:57.848349 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9aa6b-3f97-4404-867a-272d48618173 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:02:57.852674 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9aa6b-3f97-4404-867a-272d48618173 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:57.852792 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67332e9-0c4c-42c7-b557-32fd510f9acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:57.852792 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67332e9-0c4c-42c7-b557-32fd510f9acf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:57.852947 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9aa6b-3f97-4404-867a-272d48618173 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:57.853060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67332e9-0c4c-42c7-b557-32fd510f9acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:57.853060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67332e9-0c4c-42c7-b557-32fd510f9acf admin admin] PATH_INFO: `/v3/projects/1316a89a9bc7405890697a5e1cd96927` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:57.853060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67332e9-0c4c-42c7-b557-32fd510f9acf admin admin] RBAC: Authorizing `identity:delete_project(project_id=1316a89a9bc7405890697a5e1cd96927)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:57.854121 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67332e9-0c4c-42c7-b557-32fd510f9acf admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:57.863872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1051/2117] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:02:57.865054 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9aa6b-3f97-4404-867a-272d48618173 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:57.875834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9aa6b-3f97-4404-867a-272d48618173 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:57.881448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77111a3d-89e8-46b5-b07b-62fb8062a4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:57.882016 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77111a3d-89e8-46b5-b07b-62fb8062a4a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:57.882288 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77111a3d-89e8-46b5-b07b-62fb8062a4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:57.882566 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77111a3d-89e8-46b5-b07b-62fb8062a4a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:57.884724 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77111a3d-89e8-46b5-b07b-62fb8062a4a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:02:57.885035 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77111a3d-89e8-46b5-b07b-62fb8062a4a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:57.886691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77111a3d-89e8-46b5-b07b-62fb8062a4a9 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:57.901666 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1052/2118] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:02:57.902484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77111a3d-89e8-46b5-b07b-62fb8062a4a9 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:57.913075 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77111a3d-89e8-46b5-b07b-62fb8062a4a9 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:57.917943 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5b1f4-753e-442a-9706-d51156976b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:57.918514 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5b1f4-753e-442a-9706-d51156976b24 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:57.918814 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5b1f4-753e-442a-9706-d51156976b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:57.919111 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5b1f4-753e-442a-9706-d51156976b24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:57.919727 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5b1f4-753e-442a-9706-d51156976b24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:57.921628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5b1f4-753e-442a-9706-d51156976b24 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:57.977454 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b67332e9-0c4c-42c7-b557-32fd510f9acf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1316a89a9bc7405890697a5e1cd96927', 'request_id': 'req-b67332e9-0c4c-42c7-b557-32fd510f9acf'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:58.016536 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1053/2119] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:02:57 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:58.017990 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5b1f4-753e-442a-9706-d51156976b24 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:58.026756 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b67332e9-0c4c-42c7-b557-32fd510f9acf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1316a89a9bc7405890697a5e1cd96927', 'request_id': 'req-b67332e9-0c4c-42c7-b557-32fd510f9acf'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:58.032668 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b67332e9-0c4c-42c7-b557-32fd510f9acf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1316a89a9bc7405890697a5e1cd96927', 'request_id': 'req-b67332e9-0c4c-42c7-b557-32fd510f9acf'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:58.041673 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:58.043994 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b67332e9-0c4c-42c7-b557-32fd510f9acf admin admin] The token cache is being invalidate because project 1316a89a9bc7405890697a5e1cd96927 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:58.044297 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b67332e9-0c4c-42c7-b557-32fd510f9acf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:58.044955 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b67332e9-0c4c-42c7-b557-32fd510f9acf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:58.046450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1067/2120] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:02:57 2022] DELETE /identity/v3/projects/1316a89a9bc7405890697a5e1cd96927 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:58.047394 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67332e9-0c4c-42c7-b557-32fd510f9acf admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:58.049847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b67332e9-0c4c-42c7-b557-32fd510f9acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:58.085993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5b1f4-753e-442a-9706-d51156976b24 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:58.091004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53418eaf-15e9-4066-94f9-c58382f2a9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:58.091604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53418eaf-15e9-4066-94f9-c58382f2a9fc admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:58.091884 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53418eaf-15e9-4066-94f9-c58382f2a9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:58.092157 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53418eaf-15e9-4066-94f9-c58382f2a9fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:58.092784 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53418eaf-15e9-4066-94f9-c58382f2a9fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:58.094863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53418eaf-15e9-4066-94f9-c58382f2a9fc admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:58.107607 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:58.123076 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67332e9-0c4c-42c7-b557-32fd510f9acf admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:58.127927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5220837-5ace-4747-a13f-7d3732b6a3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:58.128561 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5220837-5ace-4747-a13f-7d3732b6a3ac admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:58.129460 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5220837-5ace-4747-a13f-7d3732b6a3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:58.129736 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5220837-5ace-4747-a13f-7d3732b6a3ac admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:58.130521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5220837-5ace-4747-a13f-7d3732b6a3ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:58.132509 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5220837-5ace-4747-a13f-7d3732b6a3ac admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:58.186504 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1054/2121] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:02:58 2022] POST /identity/v3/projects => generated 369 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:58.186504 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53418eaf-15e9-4066-94f9-c58382f2a9fc admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:58.186504 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-53418eaf-15e9-4066-94f9-c58382f2a9fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:58.249743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53418eaf-15e9-4066-94f9-c58382f2a9fc admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:58.254543 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b833bc99-b8be-42ae-ab20-cfc9a5723321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:58.255239 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833bc99-b8be-42ae-ab20-cfc9a5723321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:58.255524 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833bc99-b8be-42ae-ab20-cfc9a5723321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:58.255798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833bc99-b8be-42ae-ab20-cfc9a5723321 admin admin] PATH_INFO: `/v3/users/5850ffa4a1544b2cbe7a81b338f2537f` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:58.255992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1068/2122] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:02:58 2022] POST /identity/v3/users => generated 294 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:58.256868 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5220837-5ace-4747-a13f-7d3732b6a3ac admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:58.256987 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b833bc99-b8be-42ae-ab20-cfc9a5723321 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5850ffa4a1544b2cbe7a81b338f2537f)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:58.258418 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f5220837-5ace-4747-a13f-7d3732b6a3ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:58.259120 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b833bc99-b8be-42ae-ab20-cfc9a5723321 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:58.274554 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5220837-5ace-4747-a13f-7d3732b6a3ac admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:58.281447 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96148ba8-286d-4734-bbaa-eb8c6a7ff495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:58.282088 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96148ba8-286d-4734-bbaa-eb8c6a7ff495 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:58.282470 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96148ba8-286d-4734-bbaa-eb8c6a7ff495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:58.282718 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96148ba8-286d-4734-bbaa-eb8c6a7ff495 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:58.283361 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96148ba8-286d-4734-bbaa-eb8c6a7ff495 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:58.286224 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96148ba8-286d-4734-bbaa-eb8c6a7ff495 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:58.365336 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1069/2123] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:02:58 2022] POST /identity/v3/users => generated 285 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:58.366050 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96148ba8-286d-4734-bbaa-eb8c6a7ff495 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:58.376286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96148ba8-286d-4734-bbaa-eb8c6a7ff495 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:58.380968 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b9958c-aa44-4d46-8d85-1fc0967fd1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:58.381540 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b9958c-aa44-4d46-8d85-1fc0967fd1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:58.381816 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b9958c-aa44-4d46-8d85-1fc0967fd1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:58.382086 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b9958c-aa44-4d46-8d85-1fc0967fd1e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:58.382742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b9958c-aa44-4d46-8d85-1fc0967fd1e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:58.384359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b9958c-aa44-4d46-8d85-1fc0967fd1e8 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:58.395679 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1070/2124] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:58.396530 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b9958c-aa44-4d46-8d85-1fc0967fd1e8 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:58.406745 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b9958c-aa44-4d46-8d85-1fc0967fd1e8 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:58.412530 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f6992-3e3a-4a98-b9dd-2acc85a35536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:58.413181 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f6992-3e3a-4a98-b9dd-2acc85a35536 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:58.413451 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f6992-3e3a-4a98-b9dd-2acc85a35536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:58.413719 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f6992-3e3a-4a98-b9dd-2acc85a35536 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:58.414348 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f6992-3e3a-4a98-b9dd-2acc85a35536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:58.416030 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f6992-3e3a-4a98-b9dd-2acc85a35536 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:58.426588 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1071/2125] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:58.427306 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f6992-3e3a-4a98-b9dd-2acc85a35536 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:58.437728 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f6992-3e3a-4a98-b9dd-2acc85a35536 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:58.443357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065e9f8-7f3e-46cd-9e55-47d1ea397c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:58.444073 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065e9f8-7f3e-46cd-9e55-47d1ea397c37 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:58.444332 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065e9f8-7f3e-46cd-9e55-47d1ea397c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:58.444541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065e9f8-7f3e-46cd-9e55-47d1ea397c37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:58.445284 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0065e9f8-7f3e-46cd-9e55-47d1ea397c37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:58.447030 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0065e9f8-7f3e-46cd-9e55-47d1ea397c37 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:58.453668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b833bc99-b8be-42ae-ab20-cfc9a5723321 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5850ffa4a1544b2cbe7a81b338f2537f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:58.462647 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1072/2126] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:58.462647 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065e9f8-7f3e-46cd-9e55-47d1ea397c37 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:58.462897 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b833bc99-b8be-42ae-ab20-cfc9a5723321 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5850ffa4a1544b2cbe7a81b338f2537f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:58.473367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065e9f8-7f3e-46cd-9e55-47d1ea397c37 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:58.475591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b833bc99-b8be-42ae-ab20-cfc9a5723321 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5850ffa4a1544b2cbe7a81b338f2537f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:58.476429 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53031703-6c11-47ef-a646-d50a072570f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:58.477026 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53031703-6c11-47ef-a646-d50a072570f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:58.477330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53031703-6c11-47ef-a646-d50a072570f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:58.477583 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53031703-6c11-47ef-a646-d50a072570f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:58.478218 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53031703-6c11-47ef-a646-d50a072570f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:58.479832 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53031703-6c11-47ef-a646-d50a072570f9 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:58.492357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1073/2127] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:58.492357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53031703-6c11-47ef-a646-d50a072570f9 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:58.502887 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53031703-6c11-47ef-a646-d50a072570f9 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:58.507613 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e006894-583a-4bd8-86f2-603d32d50f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:58.508321 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e006894-583a-4bd8-86f2-603d32d50f1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:58.508644 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e006894-583a-4bd8-86f2-603d32d50f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:58.508911 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e006894-583a-4bd8-86f2-603d32d50f1a admin admin] PATH_INFO: `/v3/projects/05369f8ab2354af7aa0bfecb3ef97cac/users/7a06a4e02060494bbef24e0572f13440/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:58.524604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e006894-583a-4bd8-86f2-603d32d50f1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=05369f8ab2354af7aa0bfecb3ef97cac, user_id=7a06a4e02060494bbef24e0572f13440, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:58.527301 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e006894-583a-4bd8-86f2-603d32d50f1a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:58.562354 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b833bc99-b8be-42ae-ab20-cfc9a5723321 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5850ffa4a1544b2cbe7a81b338f2537f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:58.563020 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b833bc99-b8be-42ae-ab20-cfc9a5723321 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5850ffa4a1544b2cbe7a81b338f2537f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:58.569942 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1055/2128] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:02:58 2022] DELETE /identity/v3/users/5850ffa4a1544b2cbe7a81b338f2537f => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:58.570895 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b833bc99-b8be-42ae-ab20-cfc9a5723321 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:58.576405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b833bc99-b8be-42ae-ab20-cfc9a5723321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:58.600246 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1074/2129] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:58 2022] PUT /identity/v3/projects/05369f8ab2354af7aa0bfecb3ef97cac/users/7a06a4e02060494bbef24e0572f13440/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:58.611035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bc5ae-ea92-4897-95f7-06b9f6b9fd58 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:58.611562 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bc5ae-ea92-4897-95f7-06b9f6b9fd58 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:58.612032 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bc5ae-ea92-4897-95f7-06b9f6b9fd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:58.619786 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:58.664113 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-817bc5ae-ea92-4897-95f7-06b9f6b9fd58 None None] MFA Rules not processed for user `7a06a4e02060494bbef24e0572f13440`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:58.666177 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-817bc5ae-ea92-4897-95f7-06b9f6b9fd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:58.681412 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b833bc99-b8be-42ae-ab20-cfc9a5723321 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:58.749949 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba8cdb-29f0-4122-9b42-8ea515164843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:58.750610 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba8cdb-29f0-4122-9b42-8ea515164843 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:58.750928 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba8cdb-29f0-4122-9b42-8ea515164843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:58.751197 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba8cdb-29f0-4122-9b42-8ea515164843 admin admin] PATH_INFO: `/v3/projects/86dd4d9a2c924076a121716be57bc442/users/dc663fa6f7274749a9b8263ee7229e2f/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:58.752169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1075/2130] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:58.756778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817bc5ae-ea92-4897-95f7-06b9f6b9fd58 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:58.758117 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-817bc5ae-ea92-4897-95f7-06b9f6b9fd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:58.770741 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba8cdb-29f0-4122-9b42-8ea515164843 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86dd4d9a2c924076a121716be57bc442, user_id=dc663fa6f7274749a9b8263ee7229e2f, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:58.772916 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817bc5ae-ea92-4897-95f7-06b9f6b9fd58 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:58.775195 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba8cdb-29f0-4122-9b42-8ea515164843 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:58.778805 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4bfb8-f055-4ea3-b100-f572af45fb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:58.779432 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4bfb8-f055-4ea3-b100-f572af45fb58 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:58.779705 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4bfb8-f055-4ea3-b100-f572af45fb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:58.779970 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4bfb8-f055-4ea3-b100-f572af45fb58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:58.780596 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db4bfb8-f055-4ea3-b100-f572af45fb58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:58.782537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db4bfb8-f055-4ea3-b100-f572af45fb58 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:58.860003 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1056/2131] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:58 2022] PUT /identity/v3/projects/86dd4d9a2c924076a121716be57bc442/users/dc663fa6f7274749a9b8263ee7229e2f/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:58.869960 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef81032-e6a0-45ce-82c8-7cee2d5cb141 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:58.870193 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef81032-e6a0-45ce-82c8-7cee2d5cb141 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:58.870632 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef81032-e6a0-45ce-82c8-7cee2d5cb141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:58.887827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1076/2132] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:02:58 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:02:58.897658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4bfb8-f055-4ea3-b100-f572af45fb58 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:58.917845 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-6ef81032-e6a0-45ce-82c8-7cee2d5cb141 None None] MFA Rules not processed for user `dc663fa6f7274749a9b8263ee7229e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:58.919224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6ef81032-e6a0-45ce-82c8-7cee2d5cb141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:58.940835 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:59.002759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4bfb8-f055-4ea3-b100-f572af45fb58 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:59.009219 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdde706-45f0-472b-a1b4-cd1df7cfa031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:59.010311 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdde706-45f0-472b-a1b4-cd1df7cfa031 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:59.010592 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdde706-45f0-472b-a1b4-cd1df7cfa031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:59.011109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdde706-45f0-472b-a1b4-cd1df7cfa031 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:59.012334 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdde706-45f0-472b-a1b4-cd1df7cfa031 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:59.017602 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdde706-45f0-472b-a1b4-cd1df7cfa031 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:59.025285 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1057/2133] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:59.038163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef81032-e6a0-45ce-82c8-7cee2d5cb141 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:59.039550 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6ef81032-e6a0-45ce-82c8-7cee2d5cb141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:59.100297 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef81032-e6a0-45ce-82c8-7cee2d5cb141 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:59.105418 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2301b6-a1a1-45a6-9455-08ef8bd88cd9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb18199dc8b42589184042556e98983', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:59.105985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2301b6-a1a1-45a6-9455-08ef8bd88cd9 service glance] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:59.106271 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2301b6-a1a1-45a6-9455-08ef8bd88cd9 service glance] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:59.106534 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2301b6-a1a1-45a6-9455-08ef8bd88cd9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:59.116778 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2301b6-a1a1-45a6-9455-08ef8bd88cd9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:59.126042 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2301b6-a1a1-45a6-9455-08ef8bd88cd9 service glance] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:59.126335 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1077/2134] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:02:58 2022] POST /identity/v3/users => generated 299 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:02:59.126335 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdde706-45f0-472b-a1b4-cd1df7cfa031 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:59.126335 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-bcdde706-45f0-472b-a1b4-cd1df7cfa031 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:59.129178 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1058/2135] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:02:59.130026 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2301b6-a1a1-45a6-9455-08ef8bd88cd9 service glance] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:59.131493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8f2301b6-a1a1-45a6-9455-08ef8bd88cd9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:59.140916 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdde706-45f0-472b-a1b4-cd1df7cfa031 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:59.148391 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2301b6-a1a1-45a6-9455-08ef8bd88cd9 service glance] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:59.156215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de486e9-380f-48b1-9ba6-aebb673f8c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:59.156215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de486e9-380f-48b1-9ba6-aebb673f8c99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:59.156215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de486e9-380f-48b1-9ba6-aebb673f8c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:59.156215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de486e9-380f-48b1-9ba6-aebb673f8c99 admin admin] PATH_INFO: `/v3/users/057f4d33e4724c4995c27dbd5a2a9026` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:59.156892 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9effdbf2-79a8-4427-8d79-c258c781a831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:59.157535 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9effdbf2-79a8-4427-8d79-c258c781a831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:59.157967 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9effdbf2-79a8-4427-8d79-c258c781a831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:59.158913 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de486e9-380f-48b1-9ba6-aebb673f8c99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=057f4d33e4724c4995c27dbd5a2a9026)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:59.159775 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9effdbf2-79a8-4427-8d79-c258c781a831 admin admin] PATH_INFO: `/v3/projects/5ce0d0e814d94907bea01839a8914b72` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:59.162153 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de486e9-380f-48b1-9ba6-aebb673f8c99 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:59.164893 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9effdbf2-79a8-4427-8d79-c258c781a831 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ce0d0e814d94907bea01839a8914b72)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:59.166861 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9effdbf2-79a8-4427-8d79-c258c781a831 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:59.204007 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:59.326698 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8de486e9-380f-48b1-9ba6-aebb673f8c99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '057f4d33e4724c4995c27dbd5a2a9026'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:59.329252 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9effdbf2-79a8-4427-8d79-c258c781a831 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ce0d0e814d94907bea01839a8914b72', 'request_id': 'req-9effdbf2-79a8-4427-8d79-c258c781a831'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:59.339897 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9effdbf2-79a8-4427-8d79-c258c781a831 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ce0d0e814d94907bea01839a8914b72', 'request_id': 'req-9effdbf2-79a8-4427-8d79-c258c781a831'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:59.344193 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8de486e9-380f-48b1-9ba6-aebb673f8c99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '057f4d33e4724c4995c27dbd5a2a9026'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:59.348300 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9effdbf2-79a8-4427-8d79-c258c781a831 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ce0d0e814d94907bea01839a8914b72', 'request_id': 'req-9effdbf2-79a8-4427-8d79-c258c781a831'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:59.356290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8de486e9-380f-48b1-9ba6-aebb673f8c99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '057f4d33e4724c4995c27dbd5a2a9026'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:59.357295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8de486e9-380f-48b1-9ba6-aebb673f8c99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '057f4d33e4724c4995c27dbd5a2a9026'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:59.433028 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9effdbf2-79a8-4427-8d79-c258c781a831 admin admin] The token cache is being invalidate because project 5ce0d0e814d94907bea01839a8914b72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:02:59.433413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9effdbf2-79a8-4427-8d79-c258c781a831 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:59.434220 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9effdbf2-79a8-4427-8d79-c258c781a831 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:59.435904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1059/2136] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:02:59 2022] DELETE /identity/v3/projects/5ce0d0e814d94907bea01839a8914b72 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:59.436896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9effdbf2-79a8-4427-8d79-c258c781a831 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:59.438295 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9effdbf2-79a8-4427-8d79-c258c781a831 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:59.484043 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8de486e9-380f-48b1-9ba6-aebb673f8c99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '057f4d33e4724c4995c27dbd5a2a9026'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:02:59.491310 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1078/2137] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:02:59 2022] DELETE /identity/v3/users/057f4d33e4724c4995c27dbd5a2a9026 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:59.492514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab7dbe-fb8c-4663-96dd-96888c39e7ae None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:59.492797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab7dbe-fb8c-4663-96dd-96888c39e7ae None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:59.493692 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab7dbe-fb8c-4663-96dd-96888c39e7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:59.505178 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9effdbf2-79a8-4427-8d79-c258c781a831 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:59.514717 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-0dab7dbe-fb8c-4663-96dd-96888c39e7ae None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:59.516716 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0dab7dbe-fb8c-4663-96dd-96888c39e7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:59.575823 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ce152-42bd-4ece-b446-f2e7996a41dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:59.576432 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ce152-42bd-4ece-b446-f2e7996a41dd admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:59.576712 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ce152-42bd-4ece-b446-f2e7996a41dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:59.576980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ce152-42bd-4ece-b446-f2e7996a41dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:59.577624 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ce152-42bd-4ece-b446-f2e7996a41dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:59.579304 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ce152-42bd-4ece-b446-f2e7996a41dd admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:59.581076 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1079/2138] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:59.583528 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4295d15-dfd6-44b1-9fac-c7ca0c7a4aea None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:59.583818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4295d15-dfd6-44b1-9fac-c7ca0c7a4aea None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:59.584093 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4295d15-dfd6-44b1-9fac-c7ca0c7a4aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:59.590354 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1060/2139] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:02:59.591566 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c7e54-eeb4-473e-a0af-b3d656304886 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:59.591845 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c7e54-eeb4-473e-a0af-b3d656304886 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:59.592119 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c7e54-eeb4-473e-a0af-b3d656304886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:59.605617 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-d4295d15-dfd6-44b1-9fac-c7ca0c7a4aea None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:59.606890 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-d4295d15-dfd6-44b1-9fac-c7ca0c7a4aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:59.613584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5f6c7e54-eeb4-473e-a0af-b3d656304886 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:59.615192 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5f6c7e54-eeb4-473e-a0af-b3d656304886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:59.616849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1080/2140] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:59.617697 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4295d15-dfd6-44b1-9fac-c7ca0c7a4aea None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:59.627750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4295d15-dfd6-44b1-9fac-c7ca0c7a4aea None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:59.629497 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1061/2141] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:02:59.630401 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c7e54-eeb4-473e-a0af-b3d656304886 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:59.631891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5f6c7e54-eeb4-473e-a0af-b3d656304886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:59.633117 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df97b851-4b65-4f51-b71a-cd56f82f4b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:59.633897 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97b851-4b65-4f51-b71a-cd56f82f4b51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:59.634077 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97b851-4b65-4f51-b71a-cd56f82f4b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:59.634469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97b851-4b65-4f51-b71a-cd56f82f4b51 admin admin] PATH_INFO: `/v3/projects/14eca18de6a349c0b80ec0296b6f63d2/users/0fa40c2b6ffc4059b7646c888dec5cf7/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:59.642787 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:59.652398 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df97b851-4b65-4f51-b71a-cd56f82f4b51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14eca18de6a349c0b80ec0296b6f63d2, user_id=0fa40c2b6ffc4059b7646c888dec5cf7, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:59.655373 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df97b851-4b65-4f51-b71a-cd56f82f4b51 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:59.657720 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:02:59.692237 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c7e54-eeb4-473e-a0af-b3d656304886 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:59.696100 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9b86b-0338-4745-9982-2e9e4eb2fed0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:59.696696 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9b86b-0338-4745-9982-2e9e4eb2fed0 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:59.696976 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9b86b-0338-4745-9982-2e9e4eb2fed0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:59.697250 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9b86b-0338-4745-9982-2e9e4eb2fed0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:59.706627 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9b86b-0338-4745-9982-2e9e4eb2fed0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:59.709120 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9b86b-0338-4745-9982-2e9e4eb2fed0 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:59.719522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1062/2142] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:02:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:02:59.720261 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9b86b-0338-4745-9982-2e9e4eb2fed0 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:59.731144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9b86b-0338-4745-9982-2e9e4eb2fed0 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:59.736872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38b764-c03e-4439-9b0d-b7fc325739f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:59.737478 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38b764-c03e-4439-9b0d-b7fc325739f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:59.737895 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38b764-c03e-4439-9b0d-b7fc325739f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:59.738368 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38b764-c03e-4439-9b0d-b7fc325739f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:59.739459 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38b764-c03e-4439-9b0d-b7fc325739f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:02:59.739459 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38b764-c03e-4439-9b0d-b7fc325739f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:59.742071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38b764-c03e-4439-9b0d-b7fc325739f2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:59.755664 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1063/2143] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:02:59.759707 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38b764-c03e-4439-9b0d-b7fc325739f2 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:59.760932 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1081/2144] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:59 2022] PUT /identity/v3/projects/14eca18de6a349c0b80ec0296b6f63d2/users/0fa40c2b6ffc4059b7646c888dec5cf7/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:59.764564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df97b851-4b65-4f51-b71a-cd56f82f4b51 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:59.776713 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38b764-c03e-4439-9b0d-b7fc325739f2 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:59.848166 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ba484-ed70-42b9-a010-0edbfbfa5c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:59.848915 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ba484-ed70-42b9-a010-0edbfbfa5c18 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:59.849263 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ba484-ed70-42b9-a010-0edbfbfa5c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:59.849642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ba484-ed70-42b9-a010-0edbfbfa5c18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:59.850812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ba484-ed70-42b9-a010-0edbfbfa5c18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:59.853146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ba484-ed70-42b9-a010-0edbfbfa5c18 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:59.858906 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df97b851-4b65-4f51-b71a-cd56f82f4b51 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:59.867779 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6734d4d-a58c-42a2-acc1-bebc989d0fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:59.868683 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6734d4d-a58c-42a2-acc1-bebc989d0fda admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:59.868964 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6734d4d-a58c-42a2-acc1-bebc989d0fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:59.869257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6734d4d-a58c-42a2-acc1-bebc989d0fda admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:59.870443 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6734d4d-a58c-42a2-acc1-bebc989d0fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:59.872965 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6734d4d-a58c-42a2-acc1-bebc989d0fda admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:59.886628 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1082/2145] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:02:59.890621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6734d4d-a58c-42a2-acc1-bebc989d0fda admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:02:59.901163 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6734d4d-a58c-42a2-acc1-bebc989d0fda admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:02:59.906022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4126d-26ee-40ef-8013-ea8cd1321e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:02:59.906677 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4126d-26ee-40ef-8013-ea8cd1321e60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:59.906977 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4126d-26ee-40ef-8013-ea8cd1321e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:59.907250 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4126d-26ee-40ef-8013-ea8cd1321e60 admin admin] PATH_INFO: `/v3/projects/14eca18de6a349c0b80ec0296b6f63d2/users/0fa40c2b6ffc4059b7646c888dec5cf7/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:59.909151 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4126d-26ee-40ef-8013-ea8cd1321e60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14eca18de6a349c0b80ec0296b6f63d2, user_id=0fa40c2b6ffc4059b7646c888dec5cf7, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:02:59.912528 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4126d-26ee-40ef-8013-ea8cd1321e60 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:02:59.923229 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1083/2146] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:02:59 2022] PUT /identity/v3/projects/14eca18de6a349c0b80ec0296b6f63d2/users/0fa40c2b6ffc4059b7646c888dec5cf7/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:02:59.936399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d8ffd-19b2-4af9-a3c7-935c01eea1e8 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:02:59.936685 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d8ffd-19b2-4af9-a3c7-935c01eea1e8 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:02:59.936961 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d8ffd-19b2-4af9-a3c7-935c01eea1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:02:59.976281 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-1d5d8ffd-19b2-4af9-a3c7-935c01eea1e8 None None] MFA Rules not processed for user `0fa40c2b6ffc4059b7646c888dec5cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:02:59.977623 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1d5d8ffd-19b2-4af9-a3c7-935c01eea1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:02:59.984749 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1064/2147] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:02:59 2022] POST /identity/v3/projects => generated 383 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:02:59.989028 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ba484-ed70-42b9-a010-0edbfbfa5c18 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:00.089376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ba484-ed70-42b9-a010-0edbfbfa5c18 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:00.095616 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f45ee-2464-421e-98c7-c2cb67e9af29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:00.096590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f45ee-2464-421e-98c7-c2cb67e9af29 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:00.097107 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f45ee-2464-421e-98c7-c2cb67e9af29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:00.097571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f45ee-2464-421e-98c7-c2cb67e9af29 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:00.098632 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f45ee-2464-421e-98c7-c2cb67e9af29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:00.100723 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f45ee-2464-421e-98c7-c2cb67e9af29 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:00.103407 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1084/2148] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:02:59 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:00.120445 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d8ffd-19b2-4af9-a3c7-935c01eea1e8 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:00.124475 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1d5d8ffd-19b2-4af9-a3c7-935c01eea1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:00.197720 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d8ffd-19b2-4af9-a3c7-935c01eea1e8 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:00.210650 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:00.231227 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1065/2149] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:02:59 2022] POST /identity/v3/users => generated 292 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:03:00.235604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f45ee-2464-421e-98c7-c2cb67e9af29 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:00.246353 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f45ee-2464-421e-98c7-c2cb67e9af29 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:00.250219 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b87d8-73fe-4b53-bfe7-98f397c3a777 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:00.250858 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b87d8-73fe-4b53-bfe7-98f397c3a777 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:00.251139 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b87d8-73fe-4b53-bfe7-98f397c3a777 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:00.251339 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4b91a-1210-432c-a8fe-73a0c4316d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:00.251483 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b87d8-73fe-4b53-bfe7-98f397c3a777 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:00.251938 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4b91a-1210-432c-a8fe-73a0c4316d7c admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:00.252212 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4b91a-1210-432c-a8fe-73a0c4316d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:00.252478 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4b91a-1210-432c-a8fe-73a0c4316d7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:00.253403 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a4b91a-1210-432c-a8fe-73a0c4316d7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:00.255749 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a4b91a-1210-432c-a8fe-73a0c4316d7c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:00.261215 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:00.266525 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1066/2150] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:03:00.270597 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4b91a-1210-432c-a8fe-73a0c4316d7c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:00.280642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4b91a-1210-432c-a8fe-73a0c4316d7c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:00.287328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20524c2b-d073-419f-9306-cc961ad7a7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:00.287328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20524c2b-d073-419f-9306-cc961ad7a7bb admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:00.287328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20524c2b-d073-419f-9306-cc961ad7a7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:00.287328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20524c2b-d073-419f-9306-cc961ad7a7bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:00.287924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20524c2b-d073-419f-9306-cc961ad7a7bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:00.289663 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20524c2b-d073-419f-9306-cc961ad7a7bb admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:00.302282 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1067/2151] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:03:00.306142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20524c2b-d073-419f-9306-cc961ad7a7bb admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:00.318776 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20524c2b-d073-419f-9306-cc961ad7a7bb admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:00.324078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca4907-1f8b-4f1c-bafb-2959af2c79f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:00.324730 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca4907-1f8b-4f1c-bafb-2959af2c79f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:00.325003 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca4907-1f8b-4f1c-bafb-2959af2c79f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:00.325282 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca4907-1f8b-4f1c-bafb-2959af2c79f5 admin admin] PATH_INFO: `/v3/projects/e05baef1856b49668ef67ffad4e62e7f/users/69acbf5abd3742daafaf86bac806577f/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:00.333578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b87d8-73fe-4b53-bfe7-98f397c3a777 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:00.338936 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b87d8-73fe-4b53-bfe7-98f397c3a777 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:00.344607 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca4907-1f8b-4f1c-bafb-2959af2c79f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e05baef1856b49668ef67ffad4e62e7f, user_id=69acbf5abd3742daafaf86bac806577f, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:00.346093 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1085/2152] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:00 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:03:00.346861 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b87d8-73fe-4b53-bfe7-98f397c3a777 service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:00.348362 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b23b87d8-73fe-4b53-bfe7-98f397c3a777 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:00.350175 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca4907-1f8b-4f1c-bafb-2959af2c79f5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:00.364885 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b87d8-73fe-4b53-bfe7-98f397c3a777 service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:00.369786 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e131c-511e-42e4-b6ea-3bf6b77e5f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:00.370635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e131c-511e-42e4-b6ea-3bf6b77e5f53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:00.370906 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e131c-511e-42e4-b6ea-3bf6b77e5f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:00.371178 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e131c-511e-42e4-b6ea-3bf6b77e5f53 admin admin] PATH_INFO: `/v3/projects/d14dbcdf01214d10bb717a3e9727b7d6` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:00.372322 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7e131c-511e-42e4-b6ea-3bf6b77e5f53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d14dbcdf01214d10bb717a3e9727b7d6)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:00.374280 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7e131c-511e-42e4-b6ea-3bf6b77e5f53 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:00.444588 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1068/2153] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:03:00 2022] PUT /identity/v3/projects/e05baef1856b49668ef67ffad4e62e7f/users/69acbf5abd3742daafaf86bac806577f/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:03:00.446945 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca4907-1f8b-4f1c-bafb-2959af2c79f5 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:00.479137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-fe7e131c-511e-42e4-b6ea-3bf6b77e5f53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd14dbcdf01214d10bb717a3e9727b7d6', 'request_id': 'req-fe7e131c-511e-42e4-b6ea-3bf6b77e5f53'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:00.512718 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca4907-1f8b-4f1c-bafb-2959af2c79f5 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:00.547325 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-fe7e131c-511e-42e4-b6ea-3bf6b77e5f53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd14dbcdf01214d10bb717a3e9727b7d6', 'request_id': 'req-fe7e131c-511e-42e4-b6ea-3bf6b77e5f53'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:00.554239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-fe7e131c-511e-42e4-b6ea-3bf6b77e5f53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd14dbcdf01214d10bb717a3e9727b7d6', 'request_id': 'req-fe7e131c-511e-42e4-b6ea-3bf6b77e5f53'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:00.567701 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-fe7e131c-511e-42e4-b6ea-3bf6b77e5f53 admin admin] The token cache is being invalidate because project d14dbcdf01214d10bb717a3e9727b7d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:03:00.567920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-fe7e131c-511e-42e4-b6ea-3bf6b77e5f53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:00.570875 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-fe7e131c-511e-42e4-b6ea-3bf6b77e5f53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:00.571276 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1086/2154] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:03:00 2022] DELETE /identity/v3/projects/d14dbcdf01214d10bb717a3e9727b7d6 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:00.572900 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee1dd2-7cc9-4114-9ea2-5d54250e8f6a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:00.573181 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee1dd2-7cc9-4114-9ea2-5d54250e8f6a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:00.573453 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee1dd2-7cc9-4114-9ea2-5d54250e8f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:00.579962 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96b1f9-887e-43ef-b149-c368b4d7fbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:00.579962 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96b1f9-887e-43ef-b149-c368b4d7fbd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:00.579962 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96b1f9-887e-43ef-b149-c368b4d7fbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:00.579962 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96b1f9-887e-43ef-b149-c368b4d7fbd7 admin admin] PATH_INFO: `/v3/users/7a06a4e02060494bbef24e0572f13440` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:00.581128 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb96b1f9-887e-43ef-b149-c368b4d7fbd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a06a4e02060494bbef24e0572f13440)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:00.583500 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb96b1f9-887e-43ef-b149-c368b4d7fbd7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:00.606648 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-d7ee1dd2-7cc9-4114-9ea2-5d54250e8f6a None None] MFA Rules not processed for user `69acbf5abd3742daafaf86bac806577f`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:00.607944 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-d7ee1dd2-7cc9-4114-9ea2-5d54250e8f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:00.678193 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1087/2155] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:00.679470 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2ba9f-7f20-48c8-9ccb-3f02b9a5e216 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:00.679752 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2ba9f-7f20-48c8-9ccb-3f02b9a5e216 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:00.680026 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2ba9f-7f20-48c8-9ccb-3f02b9a5e216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:00.699104 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b4e2ba9f-7f20-48c8-9ccb-3f02b9a5e216 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:00.700310 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b4e2ba9f-7f20-48c8-9ccb-3f02b9a5e216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:00.708327 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1088/2156] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:00.709066 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2ba9f-7f20-48c8-9ccb-3f02b9a5e216 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:00.710270 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b4e2ba9f-7f20-48c8-9ccb-3f02b9a5e216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:00.721639 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2ba9f-7f20-48c8-9ccb-3f02b9a5e216 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:00.726975 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e60d4-59c8-41d4-a7ee-0e4c116661fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:00.728176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e60d4-59c8-41d4-a7ee-0e4c116661fd admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:00.728461 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e60d4-59c8-41d4-a7ee-0e4c116661fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:00.731869 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e60d4-59c8-41d4-a7ee-0e4c116661fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:00.731869 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e60d4-59c8-41d4-a7ee-0e4c116661fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:00.731869 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e60d4-59c8-41d4-a7ee-0e4c116661fd admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:00.765186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fb96b1f9-887e-43ef-b149-c368b4d7fbd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a06a4e02060494bbef24e0572f13440'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:00.775352 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fb96b1f9-887e-43ef-b149-c368b4d7fbd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a06a4e02060494bbef24e0572f13440'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:00.788404 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fb96b1f9-887e-43ef-b149-c368b4d7fbd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a06a4e02060494bbef24e0572f13440'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:00.819300 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1089/2157] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:03:00 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:03:00.820850 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15edd4-def7-4748-9889-8cb011b98d7a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:00.821196 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15edd4-def7-4748-9889-8cb011b98d7a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:00.821469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15edd4-def7-4748-9889-8cb011b98d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:00.843176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-2f15edd4-def7-4748-9889-8cb011b98d7a None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:00.844366 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2f15edd4-def7-4748-9889-8cb011b98d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:00.896992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:00.908665 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1090/2158] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:00.909497 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15edd4-def7-4748-9889-8cb011b98d7a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:00.920190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15edd4-def7-4748-9889-8cb011b98d7a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:00.925084 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb1c838-82d0-4fbc-8c99-934fcef07266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:00.926025 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1c838-82d0-4fbc-8c99-934fcef07266 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:00.926212 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1c838-82d0-4fbc-8c99-934fcef07266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:00.926773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1c838-82d0-4fbc-8c99-934fcef07266 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:00.927253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb1c838-82d0-4fbc-8c99-934fcef07266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:00.929405 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb1c838-82d0-4fbc-8c99-934fcef07266 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:00.939092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fb96b1f9-887e-43ef-b149-c368b4d7fbd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a06a4e02060494bbef24e0572f13440'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:00.939985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fb96b1f9-887e-43ef-b149-c368b4d7fbd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a06a4e02060494bbef24e0572f13440'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:00.947279 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1069/2159] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:03:00 2022] DELETE /identity/v3/users/7a06a4e02060494bbef24e0572f13440 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:00.953264 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96b1f9-887e-43ef-b149-c368b4d7fbd7 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:00.953264 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fb96b1f9-887e-43ef-b149-c368b4d7fbd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:00.970212 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:01.020829 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96b1f9-887e-43ef-b149-c368b4d7fbd7 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:01.025777 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1091/2160] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:03:00 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:03:01.025777 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07d612-d1d9-40ef-b1f6-5ff17c5924b2 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:01.025777 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07d612-d1d9-40ef-b1f6-5ff17c5924b2 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:01.025777 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07d612-d1d9-40ef-b1f6-5ff17c5924b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:01.036071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa495668-c9de-45ae-9d22-d8c9da7a68ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:01.036071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa495668-c9de-45ae-9d22-d8c9da7a68ce admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:01.036071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa495668-c9de-45ae-9d22-d8c9da7a68ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:01.036071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa495668-c9de-45ae-9d22-d8c9da7a68ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:01.036783 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa495668-c9de-45ae-9d22-d8c9da7a68ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:03:01.036783 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa495668-c9de-45ae-9d22-d8c9da7a68ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:01.036783 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa495668-c9de-45ae-9d22-d8c9da7a68ce admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:01.053016 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1070/2161] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:03:01.053893 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa495668-c9de-45ae-9d22-d8c9da7a68ce admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:01.055726 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-aa495668-c9de-45ae-9d22-d8c9da7a68ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:01.060233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-5f07d612-d1d9-40ef-b1f6-5ff17c5924b2 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:01.067516 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5f07d612-d1d9-40ef-b1f6-5ff17c5924b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:01.072007 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa495668-c9de-45ae-9d22-d8c9da7a68ce admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:01.080120 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1092/2162] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:01.081594 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b761d466-ff23-45d5-a102-87cdc18edeed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:01.083401 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761d466-ff23-45d5-a102-87cdc18edeed admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:01.083853 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761d466-ff23-45d5-a102-87cdc18edeed admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:01.084551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761d466-ff23-45d5-a102-87cdc18edeed admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:01.084661 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f07d612-d1d9-40ef-b1f6-5ff17c5924b2 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:01.085152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b761d466-ff23-45d5-a102-87cdc18edeed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:01.086889 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b761d466-ff23-45d5-a102-87cdc18edeed admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:01.102863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1071/2163] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:03:01.104078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b761d466-ff23-45d5-a102-87cdc18edeed admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:01.106300 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b761d466-ff23-45d5-a102-87cdc18edeed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:01.111254 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f07d612-d1d9-40ef-b1f6-5ff17c5924b2 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:01.118407 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682bbde4-47a5-4c0a-a613-6386247059bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:01.119986 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682bbde4-47a5-4c0a-a613-6386247059bd admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:01.119986 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682bbde4-47a5-4c0a-a613-6386247059bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:01.119986 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682bbde4-47a5-4c0a-a613-6386247059bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:01.119986 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682bbde4-47a5-4c0a-a613-6386247059bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:01.122321 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682bbde4-47a5-4c0a-a613-6386247059bd admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:01.172480 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b761d466-ff23-45d5-a102-87cdc18edeed admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:01.176286 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5872b3-3d70-4ecc-8e25-b7084e8daf7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:01.176920 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5872b3-3d70-4ecc-8e25-b7084e8daf7d service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:01.177199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5872b3-3d70-4ecc-8e25-b7084e8daf7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:01.177472 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5872b3-3d70-4ecc-8e25-b7084e8daf7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:01.186422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5872b3-3d70-4ecc-8e25-b7084e8daf7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:01.187891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5872b3-3d70-4ecc-8e25-b7084e8daf7d service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:01.198965 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1072/2164] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:03:01.199810 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5872b3-3d70-4ecc-8e25-b7084e8daf7d service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:01.210201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5872b3-3d70-4ecc-8e25-b7084e8daf7d service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:01.215026 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25ee65-fbeb-42ee-8567-624ef6370deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:01.215618 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25ee65-fbeb-42ee-8567-624ef6370deb admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:01.215891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25ee65-fbeb-42ee-8567-624ef6370deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:01.216162 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25ee65-fbeb-42ee-8567-624ef6370deb admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:01.216903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e25ee65-fbeb-42ee-8567-624ef6370deb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:01.218578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e25ee65-fbeb-42ee-8567-624ef6370deb admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:01.226865 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1093/2165] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:03:01 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:03:01.228762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682bbde4-47a5-4c0a-a613-6386247059bd admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:01.230360 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1073/2166] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:03:01.234174 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25ee65-fbeb-42ee-8567-624ef6370deb admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:01.246192 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25ee65-fbeb-42ee-8567-624ef6370deb admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:01.302736 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682bbde4-47a5-4c0a-a613-6386247059bd admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:01.309614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858a387-8481-4184-a1dd-0db24706f440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:01.310236 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858a387-8481-4184-a1dd-0db24706f440 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:01.310514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858a387-8481-4184-a1dd-0db24706f440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:01.310822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858a387-8481-4184-a1dd-0db24706f440 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:01.311469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858a387-8481-4184-a1dd-0db24706f440 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:01.313742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858a387-8481-4184-a1dd-0db24706f440 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:01.324122 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f02d18-afb6-4682-bb60-7cfd4c90dda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:01.324779 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f02d18-afb6-4682-bb60-7cfd4c90dda6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:01.325056 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f02d18-afb6-4682-bb60-7cfd4c90dda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:01.325325 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f02d18-afb6-4682-bb60-7cfd4c90dda6 admin admin] PATH_INFO: `/v3/projects/df3c2011ab564f0c98a8675e05ec5c7c/users/c51185609236427bbcb064b766bd2920/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:01.335471 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:01.341488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f02d18-afb6-4682-bb60-7cfd4c90dda6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df3c2011ab564f0c98a8675e05ec5c7c, user_id=c51185609236427bbcb064b766bd2920, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:01.344220 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f02d18-afb6-4682-bb60-7cfd4c90dda6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:01.405334 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1094/2167] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:03:01 2022] POST /identity/v3/users => generated 288 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:03:01.409285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858a387-8481-4184-a1dd-0db24706f440 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:01.419707 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1074/2168] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:03:01 2022] PUT /identity/v3/projects/df3c2011ab564f0c98a8675e05ec5c7c/users/c51185609236427bbcb064b766bd2920/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:03:01.421581 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47606f0-c37f-4f86-bedf-7045ea95b3ee None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:01.421878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47606f0-c37f-4f86-bedf-7045ea95b3ee None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:01.422136 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858a387-8481-4184-a1dd-0db24706f440 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:01.422443 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47606f0-c37f-4f86-bedf-7045ea95b3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:01.451017 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:01.464682 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f47606f0-c37f-4f86-bedf-7045ea95b3ee None None] MFA Rules not processed for user `c51185609236427bbcb064b766bd2920`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:01.467903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f47606f0-c37f-4f86-bedf-7045ea95b3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:01.533210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f476e8b-ae25-45d6-98eb-45e3629943c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:01.536605 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f476e8b-ae25-45d6-98eb-45e3629943c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:01.536605 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f476e8b-ae25-45d6-98eb-45e3629943c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:01.536605 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f476e8b-ae25-45d6-98eb-45e3629943c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:01.536605 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f476e8b-ae25-45d6-98eb-45e3629943c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:01.537946 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f476e8b-ae25-45d6-98eb-45e3629943c6 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:01.549610 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1095/2169] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:03:01.554238 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f476e8b-ae25-45d6-98eb-45e3629943c6 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:01.568615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f476e8b-ae25-45d6-98eb-45e3629943c6 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:01.569920 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1075/2170] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:01.577180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47606f0-c37f-4f86-bedf-7045ea95b3ee None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:01.579093 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda25cc-6219-430c-92d0-6fadadddfd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:01.579717 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda25cc-6219-430c-92d0-6fadadddfd25 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:01.580020 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda25cc-6219-430c-92d0-6fadadddfd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:01.580294 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda25cc-6219-430c-92d0-6fadadddfd25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:01.580989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecda25cc-6219-430c-92d0-6fadadddfd25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:01.583458 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecda25cc-6219-430c-92d0-6fadadddfd25 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:01.590317 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47606f0-c37f-4f86-bedf-7045ea95b3ee None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:01.594049 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a577a5-298c-4db0-bb25-cbddbb5052f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:01.594184 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1096/2171] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:03:01.594792 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a577a5-298c-4db0-bb25-cbddbb5052f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:01.595111 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a577a5-298c-4db0-bb25-cbddbb5052f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:01.595449 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a577a5-298c-4db0-bb25-cbddbb5052f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:01.596152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a577a5-298c-4db0-bb25-cbddbb5052f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:01.598991 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a577a5-298c-4db0-bb25-cbddbb5052f8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:01.599466 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda25cc-6219-430c-92d0-6fadadddfd25 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:01.609346 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda25cc-6219-430c-92d0-6fadadddfd25 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:01.614523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700a263-3a2c-4d6b-892d-797599923802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:01.619047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700a263-3a2c-4d6b-892d-797599923802 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:01.619047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700a263-3a2c-4d6b-892d-797599923802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:01.619047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700a263-3a2c-4d6b-892d-797599923802 admin admin] PATH_INFO: `/v3/projects/1b23618d7cd44d429d5923f62b9c78e6/users/87f00c68976249498c8378a828c2bc5c/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:01.630905 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5700a263-3a2c-4d6b-892d-797599923802 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b23618d7cd44d429d5923f62b9c78e6, user_id=87f00c68976249498c8378a828c2bc5c, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:01.633598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5700a263-3a2c-4d6b-892d-797599923802 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:01.726233 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1076/2172] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:03:01 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:03:01.727200 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a577a5-298c-4db0-bb25-cbddbb5052f8 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:01.729851 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-74a577a5-298c-4db0-bb25-cbddbb5052f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:01.737426 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1097/2173] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:03:01 2022] PUT /identity/v3/projects/1b23618d7cd44d429d5923f62b9c78e6/users/87f00c68976249498c8378a828c2bc5c/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:01.738475 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700a263-3a2c-4d6b-892d-797599923802 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:01.801605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a577a5-298c-4db0-bb25-cbddbb5052f8 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:01.805118 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700a263-3a2c-4d6b-892d-797599923802 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:01.806546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc1f9b-3e89-41ea-be2e-ac127b3dd831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:01.807207 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc1f9b-3e89-41ea-be2e-ac127b3dd831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:01.807488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc1f9b-3e89-41ea-be2e-ac127b3dd831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:01.807769 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc1f9b-3e89-41ea-be2e-ac127b3dd831 admin admin] PATH_INFO: `/v3/projects/05369f8ab2354af7aa0bfecb3ef97cac` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:01.808937 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc1f9b-3e89-41ea-be2e-ac127b3dd831 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05369f8ab2354af7aa0bfecb3ef97cac)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:01.809904 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0168fa7c-88e1-4a15-ba05-c86a5bc30e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:01.810494 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0168fa7c-88e1-4a15-ba05-c86a5bc30e9b admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:01.810799 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0168fa7c-88e1-4a15-ba05-c86a5bc30e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:01.810926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc1f9b-3e89-41ea-be2e-ac127b3dd831 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:01.811068 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0168fa7c-88e1-4a15-ba05-c86a5bc30e9b admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:01.811688 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0168fa7c-88e1-4a15-ba05-c86a5bc30e9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:01.813582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0168fa7c-88e1-4a15-ba05-c86a5bc30e9b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:01.908425 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1098/2174] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:03:01 2022] POST /identity/v3/users => generated 298 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:03:01.908425 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422b005-31bb-45d9-9684-4e9da4425ff9 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:01.908425 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422b005-31bb-45d9-9684-4e9da4425ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:01.908425 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422b005-31bb-45d9-9684-4e9da4425ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:01.932873 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6cfc1f9b-3e89-41ea-be2e-ac127b3dd831 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05369f8ab2354af7aa0bfecb3ef97cac', 'request_id': 'req-6cfc1f9b-3e89-41ea-be2e-ac127b3dd831'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:01.938110 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b422b005-31bb-45d9-9684-4e9da4425ff9 None None] MFA Rules not processed for user `87f00c68976249498c8378a828c2bc5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:01.943808 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6cfc1f9b-3e89-41ea-be2e-ac127b3dd831 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05369f8ab2354af7aa0bfecb3ef97cac', 'request_id': 'req-6cfc1f9b-3e89-41ea-be2e-ac127b3dd831'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:01.943932 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b422b005-31bb-45d9-9684-4e9da4425ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:01.944482 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:01.946918 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6cfc1f9b-3e89-41ea-be2e-ac127b3dd831 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05369f8ab2354af7aa0bfecb3ef97cac', 'request_id': 'req-6cfc1f9b-3e89-41ea-be2e-ac127b3dd831'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:01.974548 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:02.011879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1099/2175] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:02.013111 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422b005-31bb-45d9-9684-4e9da4425ff9 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:02.027345 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6cfc1f9b-3e89-41ea-be2e-ac127b3dd831 admin admin] The token cache is being invalidate because project 05369f8ab2354af7aa0bfecb3ef97cac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:03:02.030735 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6cfc1f9b-3e89-41ea-be2e-ac127b3dd831 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:02.030735 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6cfc1f9b-3e89-41ea-be2e-ac127b3dd831 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:02.030735 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1077/2176] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:03:01 2022] DELETE /identity/v3/projects/05369f8ab2354af7aa0bfecb3ef97cac => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:02.030735 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc1f9b-3e89-41ea-be2e-ac127b3dd831 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:02.033903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6cfc1f9b-3e89-41ea-be2e-ac127b3dd831 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:02.083101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422b005-31bb-45d9-9684-4e9da4425ff9 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:02.088548 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b359b-c437-404f-bd47-b20d2b66d50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:02.093377 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b359b-c437-404f-bd47-b20d2b66d50b admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:02.093377 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b359b-c437-404f-bd47-b20d2b66d50b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:02.093377 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b359b-c437-404f-bd47-b20d2b66d50b admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:02.093377 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87b359b-c437-404f-bd47-b20d2b66d50b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:02.093377 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87b359b-c437-404f-bd47-b20d2b66d50b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:02.101788 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1100/2177] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:03:02.103558 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b359b-c437-404f-bd47-b20d2b66d50b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:02.104251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc1f9b-3e89-41ea-be2e-ac127b3dd831 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:02.107175 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e87b359b-c437-404f-bd47-b20d2b66d50b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:02.111001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a769189-8f1d-47c5-9a4c-47c696026b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:02.111645 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a769189-8f1d-47c5-9a4c-47c696026b14 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:02.111929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a769189-8f1d-47c5-9a4c-47c696026b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:02.112203 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a769189-8f1d-47c5-9a4c-47c696026b14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:02.112837 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a769189-8f1d-47c5-9a4c-47c696026b14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:02.114976 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a769189-8f1d-47c5-9a4c-47c696026b14 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:02.123946 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b359b-c437-404f-bd47-b20d2b66d50b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:02.133176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9fc649-c2d8-4175-97e8-88a0bf1cb92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:02.134137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9fc649-c2d8-4175-97e8-88a0bf1cb92a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:02.134137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9fc649-c2d8-4175-97e8-88a0bf1cb92a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:02.134355 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9fc649-c2d8-4175-97e8-88a0bf1cb92a admin admin] PATH_INFO: `/v3/users/0fa40c2b6ffc4059b7646c888dec5cf7` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:02.135611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9fc649-c2d8-4175-97e8-88a0bf1cb92a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fa40c2b6ffc4059b7646c888dec5cf7)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:02.138318 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9fc649-c2d8-4175-97e8-88a0bf1cb92a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:02.246613 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1078/2178] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:03:02 2022] POST /identity/v3/projects => generated 375 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:03:02.247927 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a769189-8f1d-47c5-9a4c-47c696026b14 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:02.250272 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9a769189-8f1d-47c5-9a4c-47c696026b14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:02.325020 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a769189-8f1d-47c5-9a4c-47c696026b14 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:02.329803 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520c262-bd66-4338-9441-3fc622a4a2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:02.330461 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520c262-bd66-4338-9441-3fc622a4a2cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:02.330879 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520c262-bd66-4338-9441-3fc622a4a2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:02.331035 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520c262-bd66-4338-9441-3fc622a4a2cd admin admin] PATH_INFO: `/v3/projects/565a6a46bb194f979648f44767d67aaf/users/70fe30061654419aa306fdb1453299d3/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:02.346621 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5520c262-bd66-4338-9441-3fc622a4a2cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=565a6a46bb194f979648f44767d67aaf, user_id=70fe30061654419aa306fdb1453299d3, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:02.349492 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5520c262-bd66-4338-9441-3fc622a4a2cd admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:02.394129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-bb9fc649-c2d8-4175-97e8-88a0bf1cb92a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fa40c2b6ffc4059b7646c888dec5cf7'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:02.406930 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-bb9fc649-c2d8-4175-97e8-88a0bf1cb92a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fa40c2b6ffc4059b7646c888dec5cf7'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:02.414344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-bb9fc649-c2d8-4175-97e8-88a0bf1cb92a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fa40c2b6ffc4059b7646c888dec5cf7'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:02.415142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-bb9fc649-c2d8-4175-97e8-88a0bf1cb92a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fa40c2b6ffc4059b7646c888dec5cf7'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:02.450773 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1079/2179] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:03:02 2022] PUT /identity/v3/projects/565a6a46bb194f979648f44767d67aaf/users/70fe30061654419aa306fdb1453299d3/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:02.451591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520c262-bd66-4338-9441-3fc622a4a2cd admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:02.453198 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5520c262-bd66-4338-9441-3fc622a4a2cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:02.510420 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-bb9fc649-c2d8-4175-97e8-88a0bf1cb92a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fa40c2b6ffc4059b7646c888dec5cf7'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:02.516826 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1101/2180] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:03:02 2022] DELETE /identity/v3/users/0fa40c2b6ffc4059b7646c888dec5cf7 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:02.519192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9fc649-c2d8-4175-97e8-88a0bf1cb92a admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:02.519331 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:02.519420 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-bb9fc649-c2d8-4175-97e8-88a0bf1cb92a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:02.524564 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520c262-bd66-4338-9441-3fc622a4a2cd admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:02.595400 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:02.616009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca868f08-d087-45d9-88ac-8132bca892c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:02.616910 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca868f08-d087-45d9-88ac-8132bca892c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:02.617327 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca868f08-d087-45d9-88ac-8132bca892c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:02.617690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca868f08-d087-45d9-88ac-8132bca892c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:02.618333 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca868f08-d087-45d9-88ac-8132bca892c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:02.618633 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9fc649-c2d8-4175-97e8-88a0bf1cb92a admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:02.620437 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca868f08-d087-45d9-88ac-8132bca892c3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:02.623983 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283a0f2-5717-4ec6-9f1e-29e94cb56a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:02.624909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283a0f2-5717-4ec6-9f1e-29e94cb56a5b admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:02.625142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283a0f2-5717-4ec6-9f1e-29e94cb56a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:02.625646 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283a0f2-5717-4ec6-9f1e-29e94cb56a5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:02.628763 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c283a0f2-5717-4ec6-9f1e-29e94cb56a5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:02.630731 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c283a0f2-5717-4ec6-9f1e-29e94cb56a5b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:02.642252 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1102/2181] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:03:02.647318 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283a0f2-5717-4ec6-9f1e-29e94cb56a5b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:02.658326 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283a0f2-5717-4ec6-9f1e-29e94cb56a5b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:02.663876 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa2e16-3402-427b-8f85-f5888f1d6b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:02.664527 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa2e16-3402-427b-8f85-f5888f1d6b17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:02.664801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa2e16-3402-427b-8f85-f5888f1d6b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:02.665071 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa2e16-3402-427b-8f85-f5888f1d6b17 admin admin] PATH_INFO: `/v3/projects/565a6a46bb194f979648f44767d67aaf/users/70fe30061654419aa306fdb1453299d3/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:02.666992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa2e16-3402-427b-8f85-f5888f1d6b17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=565a6a46bb194f979648f44767d67aaf, user_id=70fe30061654419aa306fdb1453299d3, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:02.669675 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa2e16-3402-427b-8f85-f5888f1d6b17 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:02.680171 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1103/2182] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:03:02 2022] PUT /identity/v3/projects/565a6a46bb194f979648f44767d67aaf/users/70fe30061654419aa306fdb1453299d3/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:03:02.690149 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0baf6e-b3fb-460f-a31f-2f759b989d40 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:02.690431 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0baf6e-b3fb-460f-a31f-2f759b989d40 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:02.690726 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0baf6e-b3fb-460f-a31f-2f759b989d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:02.716586 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1080/2183] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:03:02 2022] POST /identity/v3/users => generated 288 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:03:02.720208 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-3d0baf6e-b3fb-460f-a31f-2f759b989d40 None None] MFA Rules not processed for user `70fe30061654419aa306fdb1453299d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:02.721253 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca868f08-d087-45d9-88ac-8132bca892c3 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:02.721385 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3d0baf6e-b3fb-460f-a31f-2f759b989d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:02.732810 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca868f08-d087-45d9-88ac-8132bca892c3 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:02.803944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7d00d-843d-47fe-891a-d1e85c782f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:02.804863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7d00d-843d-47fe-891a-d1e85c782f61 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:02.805041 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7d00d-843d-47fe-891a-d1e85c782f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:02.805329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7d00d-843d-47fe-891a-d1e85c782f61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:02.805969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a7d00d-843d-47fe-891a-d1e85c782f61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:02.807623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a7d00d-843d-47fe-891a-d1e85c782f61 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:02.822064 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1081/2184] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:03:02.823070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7d00d-843d-47fe-891a-d1e85c782f61 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:02.829783 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1104/2185] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:02 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:03:02.835933 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7d00d-843d-47fe-891a-d1e85c782f61 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:02.837917 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0baf6e-b3fb-460f-a31f-2f759b989d40 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:02.839396 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3d0baf6e-b3fb-460f-a31f-2f759b989d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:02.842771 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ca03c-9e5e-45b9-9724-5ba130ed4f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:02.843551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ca03c-9e5e-45b9-9724-5ba130ed4f28 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:02.843981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ca03c-9e5e-45b9-9724-5ba130ed4f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:02.844495 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ca03c-9e5e-45b9-9724-5ba130ed4f28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:02.845070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ca03c-9e5e-45b9-9724-5ba130ed4f28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:02.846775 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ca03c-9e5e-45b9-9724-5ba130ed4f28 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:02.858705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1082/2186] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:03:02.862318 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ca03c-9e5e-45b9-9724-5ba130ed4f28 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:02.873267 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ca03c-9e5e-45b9-9724-5ba130ed4f28 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:02.882158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf893b9-67fa-4afc-8e01-ee340c5fb3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:02.882158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf893b9-67fa-4afc-8e01-ee340c5fb3b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:02.882158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf893b9-67fa-4afc-8e01-ee340c5fb3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:02.882158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf893b9-67fa-4afc-8e01-ee340c5fb3b6 admin admin] PATH_INFO: `/v3/projects/610265409c564f7aac67d31edd6152f5/users/559ef77d2bce459fb75f80e4871e2434/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:02.899286 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf893b9-67fa-4afc-8e01-ee340c5fb3b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=610265409c564f7aac67d31edd6152f5, user_id=559ef77d2bce459fb75f80e4871e2434, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:02.902018 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf893b9-67fa-4afc-8e01-ee340c5fb3b6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:02.925891 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0baf6e-b3fb-460f-a31f-2f759b989d40 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:02.929894 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265e217-f188-4e6b-8552-c9cd4bd1aaf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:02.930509 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265e217-f188-4e6b-8552-c9cd4bd1aaf2 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:02.930822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265e217-f188-4e6b-8552-c9cd4bd1aaf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:02.931094 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265e217-f188-4e6b-8552-c9cd4bd1aaf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:02.932934 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c265e217-f188-4e6b-8552-c9cd4bd1aaf2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:02.998232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c265e217-f188-4e6b-8552-c9cd4bd1aaf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:02.998926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1083/2187] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:03:02 2022] PUT /identity/v3/projects/610265409c564f7aac67d31edd6152f5/users/559ef77d2bce459fb75f80e4871e2434/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:02.999697 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c265e217-f188-4e6b-8552-c9cd4bd1aaf2 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:03.010646 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1105/2188] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:03:03.010891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b65525-c811-4ab6-9145-646abe683166 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:03.011100 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b65525-c811-4ab6-9145-646abe683166 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:03.011376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b65525-c811-4ab6-9145-646abe683166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:03.035295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265e217-f188-4e6b-8552-c9cd4bd1aaf2 service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:03.037285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c265e217-f188-4e6b-8552-c9cd4bd1aaf2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:03.050192 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-e7b65525-c811-4ab6-9145-646abe683166 None None] MFA Rules not processed for user `559ef77d2bce459fb75f80e4871e2434`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:03.050417 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e7b65525-c811-4ab6-9145-646abe683166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:03.060905 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265e217-f188-4e6b-8552-c9cd4bd1aaf2 service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:03.065299 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:03.116290 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:03.137894 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2adecf-276f-417f-a2ca-be9916dce10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:03.138529 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2adecf-276f-417f-a2ca-be9916dce10b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:03.138865 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2adecf-276f-417f-a2ca-be9916dce10b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:03.139241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2adecf-276f-417f-a2ca-be9916dce10b admin admin] PATH_INFO: `/v3/projects/14eca18de6a349c0b80ec0296b6f63d2` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:03.140384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2adecf-276f-417f-a2ca-be9916dce10b admin admin] RBAC: Authorizing `identity:delete_project(project_id=14eca18de6a349c0b80ec0296b6f63d2)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:03.141600 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1084/2189] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:03:03.142303 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2adecf-276f-417f-a2ca-be9916dce10b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:03.146606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b65525-c811-4ab6-9145-646abe683166 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:03.161285 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b65525-c811-4ab6-9145-646abe683166 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:03.166249 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e268b-a400-4b23-af53-9a1e2544bdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:03.166885 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e268b-a400-4b23-af53-9a1e2544bdab admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:03.167192 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e268b-a400-4b23-af53-9a1e2544bdab admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:03.167666 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e268b-a400-4b23-af53-9a1e2544bdab admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:03.168113 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e268b-a400-4b23-af53-9a1e2544bdab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:03.172422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e268b-a400-4b23-af53-9a1e2544bdab admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:03.279323 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7a2adecf-276f-417f-a2ca-be9916dce10b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14eca18de6a349c0b80ec0296b6f63d2', 'request_id': 'req-7a2adecf-276f-417f-a2ca-be9916dce10b'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:03.289705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1085/2190] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:03:03 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:03:03.291132 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e268b-a400-4b23-af53-9a1e2544bdab admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:03.360997 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e268b-a400-4b23-af53-9a1e2544bdab admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:03.362619 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7a2adecf-276f-417f-a2ca-be9916dce10b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14eca18de6a349c0b80ec0296b6f63d2', 'request_id': 'req-7a2adecf-276f-417f-a2ca-be9916dce10b'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:03.367167 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cbccc-1eb4-48e2-b9ed-b75f4b01c179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:03.371322 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cbccc-1eb4-48e2-b9ed-b75f4b01c179 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:03.371322 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cbccc-1eb4-48e2-b9ed-b75f4b01c179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:03.371322 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cbccc-1eb4-48e2-b9ed-b75f4b01c179 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:03.371322 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18cbccc-1eb4-48e2-b9ed-b75f4b01c179 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:03.371322 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18cbccc-1eb4-48e2-b9ed-b75f4b01c179 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:03.376448 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7a2adecf-276f-417f-a2ca-be9916dce10b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14eca18de6a349c0b80ec0296b6f63d2', 'request_id': 'req-7a2adecf-276f-417f-a2ca-be9916dce10b'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:03.389387 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7a2adecf-276f-417f-a2ca-be9916dce10b admin admin] The token cache is being invalidate because project 14eca18de6a349c0b80ec0296b6f63d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:03:03.389675 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7a2adecf-276f-417f-a2ca-be9916dce10b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:03.390633 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7a2adecf-276f-417f-a2ca-be9916dce10b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:03.392515 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1106/2191] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:03:03 2022] DELETE /identity/v3/projects/14eca18de6a349c0b80ec0296b6f63d2 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:03.398583 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7650e09-fe87-4973-83be-a79825b07de7 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:03.399529 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7650e09-fe87-4973-83be-a79825b07de7 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:03.399529 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7650e09-fe87-4973-83be-a79825b07de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:03.430912 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-e7650e09-fe87-4973-83be-a79825b07de7 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:03.432447 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e7650e09-fe87-4973-83be-a79825b07de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:03.445173 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1086/2192] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:03:03 2022] POST /identity/v3/users => generated 295 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:03:03.448211 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1107/2193] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:03.449260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cbccc-1eb4-48e2-b9ed-b75f4b01c179 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:03.450936 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f18cbccc-1eb4-48e2-b9ed-b75f4b01c179 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:03.455692 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bfe44f-8ce7-4982-8df9-08f7d2382c04 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:03.455967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bfe44f-8ce7-4982-8df9-08f7d2382c04 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:03.456235 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bfe44f-8ce7-4982-8df9-08f7d2382c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:03.471938 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cbccc-1eb4-48e2-b9ed-b75f4b01c179 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:03.479577 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-83bfe44f-8ce7-4982-8df9-08f7d2382c04 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:03.480816 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-83bfe44f-8ce7-4982-8df9-08f7d2382c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:03.487387 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61504ff6-4152-4536-aa5d-d2c705f0b875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:03.492498 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61504ff6-4152-4536-aa5d-d2c705f0b875 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:03.492498 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61504ff6-4152-4536-aa5d-d2c705f0b875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:03.492498 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61504ff6-4152-4536-aa5d-d2c705f0b875 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:03.492498 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61504ff6-4152-4536-aa5d-d2c705f0b875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:03.492498 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61504ff6-4152-4536-aa5d-d2c705f0b875 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:03.494597 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1108/2194] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:03:03.498285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bfe44f-8ce7-4982-8df9-08f7d2382c04 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:03.505888 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1087/2195] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:03:03.510052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61504ff6-4152-4536-aa5d-d2c705f0b875 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:03.525361 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61504ff6-4152-4536-aa5d-d2c705f0b875 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:03.526280 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bfe44f-8ce7-4982-8df9-08f7d2382c04 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:03.533108 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b4671-93ac-4a70-8761-080e6626e882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:03.533822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b4671-93ac-4a70-8761-080e6626e882 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:03.534176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b4671-93ac-4a70-8761-080e6626e882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:03.534364 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff04bd2-d278-421c-9726-db8635471862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:03.534482 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b4671-93ac-4a70-8761-080e6626e882 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:03.535092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff04bd2-d278-421c-9726-db8635471862 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:03.535417 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff04bd2-d278-421c-9726-db8635471862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:03.535566 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b4671-93ac-4a70-8761-080e6626e882 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:03:03.535670 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff04bd2-d278-421c-9726-db8635471862 admin admin] PATH_INFO: `/v3/projects/06bb3edd4bcf4961b06317fe3e509209/users/464947ee8e2746b8a5f48333f36fa007/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:03.536178 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b4671-93ac-4a70-8761-080e6626e882 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:03.537874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b4671-93ac-4a70-8761-080e6626e882 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:03.550763 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1109/2196] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:03:03.554034 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff04bd2-d278-421c-9726-db8635471862 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06bb3edd4bcf4961b06317fe3e509209, user_id=464947ee8e2746b8a5f48333f36fa007, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:03.555560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b4671-93ac-4a70-8761-080e6626e882 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:03.557364 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff04bd2-d278-421c-9726-db8635471862 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:03.570796 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b4671-93ac-4a70-8761-080e6626e882 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:03.576896 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d82b8-a488-4285-8ad3-16371d36c9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:03.577519 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d82b8-a488-4285-8ad3-16371d36c9ea admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:03.577804 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d82b8-a488-4285-8ad3-16371d36c9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:03.578078 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d82b8-a488-4285-8ad3-16371d36c9ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:03.578887 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d82b8-a488-4285-8ad3-16371d36c9ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:03.580817 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d82b8-a488-4285-8ad3-16371d36c9ea admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:03.621460 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1088/2197] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:03:03 2022] PUT /identity/v3/projects/06bb3edd4bcf4961b06317fe3e509209/users/464947ee8e2746b8a5f48333f36fa007/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:03:03.625164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff04bd2-d278-421c-9726-db8635471862 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:03.693448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff04bd2-d278-421c-9726-db8635471862 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:03.698695 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1110/2198] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:03:03 2022] POST /identity/v3/projects => generated 387 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:03:03.699926 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d82b8-a488-4285-8ad3-16371d36c9ea admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:03.713157 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:03.764828 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd432020-8ada-466b-adf6-5e1cc76eee0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:03.765475 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd432020-8ada-466b-adf6-5e1cc76eee0c admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:03.765762 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd432020-8ada-466b-adf6-5e1cc76eee0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:03.766088 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd432020-8ada-466b-adf6-5e1cc76eee0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:03.766707 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd432020-8ada-466b-adf6-5e1cc76eee0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:03.768328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd432020-8ada-466b-adf6-5e1cc76eee0c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:03.776390 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:03.782311 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1089/2199] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:03:03.783298 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd432020-8ada-466b-adf6-5e1cc76eee0c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:03.793204 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d82b8-a488-4285-8ad3-16371d36c9ea admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:03.794141 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd432020-8ada-466b-adf6-5e1cc76eee0c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:03.799716 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998efb9-99dd-4d26-847d-bd1dd35de161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:03.800407 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998efb9-99dd-4d26-847d-bd1dd35de161 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:03.801471 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998efb9-99dd-4d26-847d-bd1dd35de161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:03.801685 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998efb9-99dd-4d26-847d-bd1dd35de161 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:03.802473 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998efb9-99dd-4d26-847d-bd1dd35de161 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:03.804426 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998efb9-99dd-4d26-847d-bd1dd35de161 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:03.812063 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671cbc53-cc2c-4b72-af60-699f0fe87d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:03.812748 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cbc53-cc2c-4b72-af60-699f0fe87d8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:03.813024 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cbc53-cc2c-4b72-af60-699f0fe87d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:03.813299 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cbc53-cc2c-4b72-af60-699f0fe87d8b admin admin] PATH_INFO: `/v3/projects/06bb3edd4bcf4961b06317fe3e509209/users/464947ee8e2746b8a5f48333f36fa007/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:03.815286 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671cbc53-cc2c-4b72-af60-699f0fe87d8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=06bb3edd4bcf4961b06317fe3e509209, user_id=464947ee8e2746b8a5f48333f36fa007, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:03.818412 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671cbc53-cc2c-4b72-af60-699f0fe87d8b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:03.830698 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1090/2200] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:03:03 2022] PUT /identity/v3/projects/06bb3edd4bcf4961b06317fe3e509209/users/464947ee8e2746b8a5f48333f36fa007/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:03.840550 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd13f6c-5a13-41e9-899e-c55967bd72c5 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:03.840841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd13f6c-5a13-41e9-899e-c55967bd72c5 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:03.841210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd13f6c-5a13-41e9-899e-c55967bd72c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:03.872848 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-fbd13f6c-5a13-41e9-899e-c55967bd72c5 None None] MFA Rules not processed for user `464947ee8e2746b8a5f48333f36fa007`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:03.875279 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fbd13f6c-5a13-41e9-899e-c55967bd72c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:03.905415 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1111/2201] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:03:03 2022] POST /identity/v3/users => generated 294 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:03:03.909531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998efb9-99dd-4d26-847d-bd1dd35de161 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:03.919943 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998efb9-99dd-4d26-847d-bd1dd35de161 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:03.961512 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1091/2202] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:03:03.972932 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd13f6c-5a13-41e9-899e-c55967bd72c5 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:03.975113 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fbd13f6c-5a13-41e9-899e-c55967bd72c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:03.986398 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680625f4-cfb5-4730-9c5e-14688e6f72bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:03.987039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680625f4-cfb5-4730-9c5e-14688e6f72bc admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:03.987331 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680625f4-cfb5-4730-9c5e-14688e6f72bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:03.987687 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680625f4-cfb5-4730-9c5e-14688e6f72bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:03.988873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680625f4-cfb5-4730-9c5e-14688e6f72bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:03.990730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680625f4-cfb5-4730-9c5e-14688e6f72bc admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:04.001561 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1112/2203] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:03:04.006197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680625f4-cfb5-4730-9c5e-14688e6f72bc admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:04.017558 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680625f4-cfb5-4730-9c5e-14688e6f72bc admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:04.022515 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14653e0a-a3b7-4d82-8f16-4566a32f9d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:04.023166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14653e0a-a3b7-4d82-8f16-4566a32f9d5c admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:04.023442 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14653e0a-a3b7-4d82-8f16-4566a32f9d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:04.023709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14653e0a-a3b7-4d82-8f16-4566a32f9d5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:04.024340 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14653e0a-a3b7-4d82-8f16-4566a32f9d5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:04.025956 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14653e0a-a3b7-4d82-8f16-4566a32f9d5c admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:04.036764 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1113/2204] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:03:04.041958 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14653e0a-a3b7-4d82-8f16-4566a32f9d5c admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:04.044946 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd13f6c-5a13-41e9-899e-c55967bd72c5 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:04.049380 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e4dd0-ab01-45f0-a23c-a946d97f1a3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:04.050098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e4dd0-ab01-45f0-a23c-a946d97f1a3d service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:04.051138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e4dd0-ab01-45f0-a23c-a946d97f1a3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:04.051419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e4dd0-ab01-45f0-a23c-a946d97f1a3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:04.053373 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14653e0a-a3b7-4d82-8f16-4566a32f9d5c admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:04.053506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d64e4dd0-ab01-45f0-a23c-a946d97f1a3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:04.058175 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa3c98-3129-4c25-aa2c-9a6527c3b840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:04.064999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa3c98-3129-4c25-aa2c-9a6527c3b840 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:04.064999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa3c98-3129-4c25-aa2c-9a6527c3b840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:04.064999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa3c98-3129-4c25-aa2c-9a6527c3b840 admin admin] PATH_INFO: `/v3/projects/e83efb97631f4a8a9d64a2b871f03259/users/70d99257f26e44979e4ed2d9013ebbf9/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:04.082648 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa3c98-3129-4c25-aa2c-9a6527c3b840 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e83efb97631f4a8a9d64a2b871f03259, user_id=70d99257f26e44979e4ed2d9013ebbf9, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:04.083593 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa3c98-3129-4c25-aa2c-9a6527c3b840 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:04.117109 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:04.132703 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e4dd0-ab01-45f0-a23c-a946d97f1a3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:04.134246 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e4dd0-ab01-45f0-a23c-a946d97f1a3d service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:04.143838 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1092/2205] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:03:04.177437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1114/2206] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:03:04 2022] PUT /identity/v3/projects/e83efb97631f4a8a9d64a2b871f03259/users/70d99257f26e44979e4ed2d9013ebbf9/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:03:04.188443 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205476c-8c84-4b89-a177-b35a7432dfd7 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:04.188732 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205476c-8c84-4b89-a177-b35a7432dfd7 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:04.189011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205476c-8c84-4b89-a177-b35a7432dfd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:04.219230 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-3205476c-8c84-4b89-a177-b35a7432dfd7 None None] MFA Rules not processed for user `70d99257f26e44979e4ed2d9013ebbf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:04.220592 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3205476c-8c84-4b89-a177-b35a7432dfd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:04.297352 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1093/2207] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:04.302161 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa3c98-3129-4c25-aa2c-9a6527c3b840 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:04.365388 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa3c98-3129-4c25-aa2c-9a6527c3b840 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:04.371440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9cc29-bb12-465a-a4a8-01cc256f6219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:04.372078 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9cc29-bb12-465a-a4a8-01cc256f6219 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:04.372360 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9cc29-bb12-465a-a4a8-01cc256f6219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:04.372635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9cc29-bb12-465a-a4a8-01cc256f6219 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:04.373280 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9cc29-bb12-465a-a4a8-01cc256f6219 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:04.375571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9cc29-bb12-465a-a4a8-01cc256f6219 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:04.468545 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1115/2208] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:03:04 2022] POST /identity/v3/projects => generated 389 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:03:04.474077 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205476c-8c84-4b89-a177-b35a7432dfd7 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:04.485034 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205476c-8c84-4b89-a177-b35a7432dfd7 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:04.545068 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e5b67-2a82-4259-8699-a861f44fb07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:04.545740 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e5b67-2a82-4259-8699-a861f44fb07d admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:04.545954 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e5b67-2a82-4259-8699-a861f44fb07d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:04.546254 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e5b67-2a82-4259-8699-a861f44fb07d admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:04.546860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e5b67-2a82-4259-8699-a861f44fb07d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:04.548784 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e5b67-2a82-4259-8699-a861f44fb07d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:04.646348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1094/2209] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:03:04 2022] POST /identity/v3/users => generated 301 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:03:04.650156 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9cc29-bb12-465a-a4a8-01cc256f6219 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:04.660936 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9cc29-bb12-465a-a4a8-01cc256f6219 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:04.667166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4779a9-d992-442d-ac1e-213f53419078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:04.667811 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4779a9-d992-442d-ac1e-213f53419078 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:04.668115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4779a9-d992-442d-ac1e-213f53419078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:04.668363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4779a9-d992-442d-ac1e-213f53419078 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:04.669010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4779a9-d992-442d-ac1e-213f53419078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:04.670606 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4779a9-d992-442d-ac1e-213f53419078 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:04.682913 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1116/2210] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:03:04.691052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e5b67-2a82-4259-8699-a861f44fb07d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:04.704822 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e5b67-2a82-4259-8699-a861f44fb07d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:04.710934 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce35512-8e0b-47c5-8412-a4ad7b45e7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:04.711659 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce35512-8e0b-47c5-8412-a4ad7b45e7e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:04.711942 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce35512-8e0b-47c5-8412-a4ad7b45e7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:04.716051 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce35512-8e0b-47c5-8412-a4ad7b45e7e5 admin admin] PATH_INFO: `/v3/projects/9e9e66c4d5e54c9dbb06d55bc4d98136/users/ed54b099b82746919b7761695a143196/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:04.749638 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce35512-8e0b-47c5-8412-a4ad7b45e7e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e9e66c4d5e54c9dbb06d55bc4d98136, user_id=ed54b099b82746919b7761695a143196, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:04.752875 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce35512-8e0b-47c5-8412-a4ad7b45e7e5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:04.871723 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1095/2211] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:03:04 2022] PUT /identity/v3/projects/9e9e66c4d5e54c9dbb06d55bc4d98136/users/ed54b099b82746919b7761695a143196/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:03:04.875788 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4779a9-d992-442d-ac1e-213f53419078 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:04.887150 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4779a9-d992-442d-ac1e-213f53419078 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:04.897576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:04.952210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091f6f1-6094-4a56-9e2f-6a0f772bb101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:04.952892 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091f6f1-6094-4a56-9e2f-6a0f772bb101 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:04.953176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091f6f1-6094-4a56-9e2f-6a0f772bb101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:04.953453 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091f6f1-6094-4a56-9e2f-6a0f772bb101 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:04.954143 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e091f6f1-6094-4a56-9e2f-6a0f772bb101 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:04.956094 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e091f6f1-6094-4a56-9e2f-6a0f772bb101 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:04.967256 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1117/2212] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:03:04.972915 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce35512-8e0b-47c5-8412-a4ad7b45e7e5 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:04.982902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce35512-8e0b-47c5-8412-a4ad7b45e7e5 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:04.987427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae4dbc-8a35-4524-9d8b-c41fad347841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:04.988730 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae4dbc-8a35-4524-9d8b-c41fad347841 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:04.989017 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae4dbc-8a35-4524-9d8b-c41fad347841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:04.989370 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae4dbc-8a35-4524-9d8b-c41fad347841 admin admin] PATH_INFO: `/v3/projects/9e9e66c4d5e54c9dbb06d55bc4d98136/users/ed54b099b82746919b7761695a143196/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:04.991493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae4dbc-8a35-4524-9d8b-c41fad347841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e9e66c4d5e54c9dbb06d55bc4d98136, user_id=ed54b099b82746919b7761695a143196, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:04.994346 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae4dbc-8a35-4524-9d8b-c41fad347841 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:05.007597 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1096/2213] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:03:04 2022] PUT /identity/v3/projects/9e9e66c4d5e54c9dbb06d55bc4d98136/users/ed54b099b82746919b7761695a143196/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:05.019830 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8e88b-a053-4494-98b5-b83975515b28 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:05.020539 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8e88b-a053-4494-98b5-b83975515b28 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:05.021010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8e88b-a053-4494-98b5-b83975515b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:05.054867 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-f3b8e88b-a053-4494-98b5-b83975515b28 None None] MFA Rules not processed for user `ed54b099b82746919b7761695a143196`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:05.056127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f3b8e88b-a053-4494-98b5-b83975515b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:05.138401 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1118/2214] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:05 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:05.149278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae4dbc-8a35-4524-9d8b-c41fad347841 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:05.151020 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-58ae4dbc-8a35-4524-9d8b-c41fad347841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:05.154736 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8e88b-a053-4494-98b5-b83975515b28 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:05.183780 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:05.211643 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae4dbc-8a35-4524-9d8b-c41fad347841 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:05.212680 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8e88b-a053-4494-98b5-b83975515b28 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:05.215411 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7b7d9-c087-4a8c-95e0-5577540f85dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:05.216013 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7b7d9-c087-4a8c-95e0-5577540f85dd service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:05.216229 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3fd9b-269d-4f43-b9f4-95f44ca20715 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:05.216348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7b7d9-c087-4a8c-95e0-5577540f85dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:05.216547 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7b7d9-c087-4a8c-95e0-5577540f85dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:05.216820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3fd9b-269d-4f43-b9f4-95f44ca20715 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:05.217090 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3fd9b-269d-4f43-b9f4-95f44ca20715 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:05.217355 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3fd9b-269d-4f43-b9f4-95f44ca20715 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:05.219372 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-57d3fd9b-269d-4f43-b9f4-95f44ca20715 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:05.227581 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d7b7d9-c087-4a8c-95e0-5577540f85dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:05.229165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d7b7d9-c087-4a8c-95e0-5577540f85dd service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:05.240113 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1097/2215] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:05 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:03:05.275556 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:05.288136 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3fd9b-269d-4f43-b9f4-95f44ca20715 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:05.289577 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3fd9b-269d-4f43-b9f4-95f44ca20715 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:05.307280 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1119/2216] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:03:05.426517 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7b7d9-c087-4a8c-95e0-5577540f85dd service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:05.490525 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7b7d9-c087-4a8c-95e0-5577540f85dd service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:05.497044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118cc37-01e8-4b8b-8617-d53ca85779f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:05.497986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118cc37-01e8-4b8b-8617-d53ca85779f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:05.498105 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118cc37-01e8-4b8b-8617-d53ca85779f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:05.498352 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118cc37-01e8-4b8b-8617-d53ca85779f8 admin admin] PATH_INFO: `/v3/users/70d99257f26e44979e4ed2d9013ebbf9` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:05.499592 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2118cc37-01e8-4b8b-8617-d53ca85779f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70d99257f26e44979e4ed2d9013ebbf9)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:05.501691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2118cc37-01e8-4b8b-8617-d53ca85779f8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:05.680931 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2118cc37-01e8-4b8b-8617-d53ca85779f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70d99257f26e44979e4ed2d9013ebbf9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:05.687739 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2118cc37-01e8-4b8b-8617-d53ca85779f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70d99257f26e44979e4ed2d9013ebbf9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:05.701434 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2118cc37-01e8-4b8b-8617-d53ca85779f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70d99257f26e44979e4ed2d9013ebbf9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:05.809151 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2118cc37-01e8-4b8b-8617-d53ca85779f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70d99257f26e44979e4ed2d9013ebbf9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:05.809912 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2118cc37-01e8-4b8b-8617-d53ca85779f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70d99257f26e44979e4ed2d9013ebbf9'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:05.819426 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1098/2217] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:03:05 2022] DELETE /identity/v3/users/70d99257f26e44979e4ed2d9013ebbf9 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:05.828199 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382a6c3-6b86-4fa9-9e5a-d171c933dcd7 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:05.828691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382a6c3-6b86-4fa9-9e5a-d171c933dcd7 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:05.830642 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382a6c3-6b86-4fa9-9e5a-d171c933dcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:05.856520 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-d382a6c3-6b86-4fa9-9e5a-d171c933dcd7 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:05.858205 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-d382a6c3-6b86-4fa9-9e5a-d171c933dcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:05.943227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1120/2218] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:03:05.950250 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118cc37-01e8-4b8b-8617-d53ca85779f8 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:05.952108 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2118cc37-01e8-4b8b-8617-d53ca85779f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:06.031788 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:06.036669 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118cc37-01e8-4b8b-8617-d53ca85779f8 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:06.040557 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e6028-411b-4b2e-b37a-38e086ba20a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:06.041295 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e6028-411b-4b2e-b37a-38e086ba20a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:06.041671 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e6028-411b-4b2e-b37a-38e086ba20a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:06.042065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e6028-411b-4b2e-b37a-38e086ba20a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:06.055767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4e6028-411b-4b2e-b37a-38e086ba20a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:06.058696 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4e6028-411b-4b2e-b37a-38e086ba20a4 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:06.068427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1099/2219] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:03:06.505954 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382a6c3-6b86-4fa9-9e5a-d171c933dcd7 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:06.509029 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-d382a6c3-6b86-4fa9-9e5a-d171c933dcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:06.538020 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382a6c3-6b86-4fa9-9e5a-d171c933dcd7 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:06.541870 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67b865-1be8-4867-b136-496797f43415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:06.542637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67b865-1be8-4867-b136-496797f43415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:06.543063 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67b865-1be8-4867-b136-496797f43415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:06.543377 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67b865-1be8-4867-b136-496797f43415 admin admin] PATH_INFO: `/v3/projects/e83efb97631f4a8a9d64a2b871f03259` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:06.546191 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c67b865-1be8-4867-b136-496797f43415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e83efb97631f4a8a9d64a2b871f03259)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:06.548340 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c67b865-1be8-4867-b136-496797f43415 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:06.715396 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0c67b865-1be8-4867-b136-496797f43415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e83efb97631f4a8a9d64a2b871f03259', 'request_id': 'req-0c67b865-1be8-4867-b136-496797f43415'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:06.833076 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0c67b865-1be8-4867-b136-496797f43415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e83efb97631f4a8a9d64a2b871f03259', 'request_id': 'req-0c67b865-1be8-4867-b136-496797f43415'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:06.848931 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0c67b865-1be8-4867-b136-496797f43415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e83efb97631f4a8a9d64a2b871f03259', 'request_id': 'req-0c67b865-1be8-4867-b136-496797f43415'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:06.874529 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e6028-411b-4b2e-b37a-38e086ba20a4 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:06.875063 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0c67b865-1be8-4867-b136-496797f43415 admin admin] The token cache is being invalidate because project e83efb97631f4a8a9d64a2b871f03259 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:03:06.875063 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0c67b865-1be8-4867-b136-496797f43415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:06.875757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0c67b865-1be8-4867-b136-496797f43415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:06.877214 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7c4e6028-411b-4b2e-b37a-38e086ba20a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:06.877788 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1121/2220] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:03:06 2022] DELETE /identity/v3/projects/e83efb97631f4a8a9d64a2b871f03259 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:06.881725 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67b865-1be8-4867-b136-496797f43415 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:06.883323 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0c67b865-1be8-4867-b136-496797f43415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:06.946366 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e6028-411b-4b2e-b37a-38e086ba20a4 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:06.963043 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67b865-1be8-4867-b136-496797f43415 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:06.968006 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c8290-2b11-48c4-b251-9a0b5a4e501e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:06.968862 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c8290-2b11-48c4-b251-9a0b5a4e501e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:06.969140 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c8290-2b11-48c4-b251-9a0b5a4e501e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:06.969409 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c8290-2b11-48c4-b251-9a0b5a4e501e admin admin] PATH_INFO: `/v3/users/ed54b099b82746919b7761695a143196` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:06.970528 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c8290-2b11-48c4-b251-9a0b5a4e501e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed54b099b82746919b7761695a143196)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:06.972564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c8290-2b11-48c4-b251-9a0b5a4e501e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:06.987480 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:06.999285 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c64ec03-fd03-438f-967b-321e24fc07ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:06.999886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64ec03-fd03-438f-967b-321e24fc07ba service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:07.000167 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64ec03-fd03-438f-967b-321e24fc07ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:07.000437 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64ec03-fd03-438f-967b-321e24fc07ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:07.002181 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9c64ec03-fd03-438f-967b-321e24fc07ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:07.061711 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c64ec03-fd03-438f-967b-321e24fc07ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:07.063195 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c64ec03-fd03-438f-967b-321e24fc07ba service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:07.073758 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1100/2221] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:03:07.074852 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c64ec03-fd03-438f-967b-321e24fc07ba service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:07.076852 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9c64ec03-fd03-438f-967b-321e24fc07ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:07.088594 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c64ec03-fd03-438f-967b-321e24fc07ba service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:07.092391 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec381dd-cd42-417b-8e5b-d7aab579cab7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:07.092990 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec381dd-cd42-417b-8e5b-d7aab579cab7 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:07.093292 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec381dd-cd42-417b-8e5b-d7aab579cab7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:07.093858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec381dd-cd42-417b-8e5b-d7aab579cab7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:07.096101 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-cec381dd-cd42-417b-8e5b-d7aab579cab7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:07.156736 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-196c8290-2b11-48c4-b251-9a0b5a4e501e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed54b099b82746919b7761695a143196'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:07.173149 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-196c8290-2b11-48c4-b251-9a0b5a4e501e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed54b099b82746919b7761695a143196'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:07.189043 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-196c8290-2b11-48c4-b251-9a0b5a4e501e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed54b099b82746919b7761695a143196'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:07.190254 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-196c8290-2b11-48c4-b251-9a0b5a4e501e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed54b099b82746919b7761695a143196'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:07.193748 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:07.197508 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec381dd-cd42-417b-8e5b-d7aab579cab7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:07.199380 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec381dd-cd42-417b-8e5b-d7aab579cab7 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:07.211412 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1101/2222] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:07 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:03:07.310842 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-196c8290-2b11-48c4-b251-9a0b5a4e501e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed54b099b82746919b7761695a143196'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:07.318550 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1122/2223] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:03:06 2022] DELETE /identity/v3/users/ed54b099b82746919b7761695a143196 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:03:07.773376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec381dd-cd42-417b-8e5b-d7aab579cab7 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:07.787853 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec381dd-cd42-417b-8e5b-d7aab579cab7 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:07.853214 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd85d1e-c2b3-4662-837b-ee044836ab9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:07.853848 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd85d1e-c2b3-4662-837b-ee044836ab9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:07.854152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd85d1e-c2b3-4662-837b-ee044836ab9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:07.854428 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd85d1e-c2b3-4662-837b-ee044836ab9f admin admin] PATH_INFO: `/v3/projects/9e9e66c4d5e54c9dbb06d55bc4d98136` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:07.855582 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd85d1e-c2b3-4662-837b-ee044836ab9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e9e66c4d5e54c9dbb06d55bc4d98136)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:07.857522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd85d1e-c2b3-4662-837b-ee044836ab9f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:07.992385 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-efd85d1e-c2b3-4662-837b-ee044836ab9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e9e66c4d5e54c9dbb06d55bc4d98136', 'request_id': 'req-efd85d1e-c2b3-4662-837b-ee044836ab9f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:07.998512 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-efd85d1e-c2b3-4662-837b-ee044836ab9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e9e66c4d5e54c9dbb06d55bc4d98136', 'request_id': 'req-efd85d1e-c2b3-4662-837b-ee044836ab9f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:08.004859 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-efd85d1e-c2b3-4662-837b-ee044836ab9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e9e66c4d5e54c9dbb06d55bc4d98136', 'request_id': 'req-efd85d1e-c2b3-4662-837b-ee044836ab9f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:08.113856 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-efd85d1e-c2b3-4662-837b-ee044836ab9f admin admin] The token cache is being invalidate because project 9e9e66c4d5e54c9dbb06d55bc4d98136 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:03:08.114167 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-efd85d1e-c2b3-4662-837b-ee044836ab9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:08.117814 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-efd85d1e-c2b3-4662-837b-ee044836ab9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:08.120492 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1102/2224] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:03:07 2022] DELETE /identity/v3/projects/9e9e66c4d5e54c9dbb06d55bc4d98136 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:03:08.140160 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdb35f-a990-43a4-b2c4-cdd50b8e6fef None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:08.140160 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdb35f-a990-43a4-b2c4-cdd50b8e6fef None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:08.140160 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdb35f-a990-43a4-b2c4-cdd50b8e6fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:08.159321 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-dfbdb35f-a990-43a4-b2c4-cdd50b8e6fef None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:08.161171 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-dfbdb35f-a990-43a4-b2c4-cdd50b8e6fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:08.245167 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1123/2225] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:08.252570 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b05b923-b26a-4787-ba78-dc97123d678a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:08.252909 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b05b923-b26a-4787-ba78-dc97123d678a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:08.253191 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b05b923-b26a-4787-ba78-dc97123d678a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:08.277326 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-4b05b923-b26a-4787-ba78-dc97123d678a None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:08.277824 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4b05b923-b26a-4787-ba78-dc97123d678a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:08.286384 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1103/2226] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:08.291248 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbdb35f-a990-43a4-b2c4-cdd50b8e6fef None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:08.324845 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbdb35f-a990-43a4-b2c4-cdd50b8e6fef None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:08.333173 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d92869-f954-4c3d-ab6e-837d95fb6b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:08.336097 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d92869-f954-4c3d-ab6e-837d95fb6b21 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:08.336666 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d92869-f954-4c3d-ab6e-837d95fb6b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:08.337013 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d92869-f954-4c3d-ab6e-837d95fb6b21 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:08.337761 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d92869-f954-4c3d-ab6e-837d95fb6b21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:03:08.338139 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d92869-f954-4c3d-ab6e-837d95fb6b21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:08.340821 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d92869-f954-4c3d-ab6e-837d95fb6b21 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:08.363931 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1124/2227] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:03:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:03:08.368357 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b05b923-b26a-4787-ba78-dc97123d678a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:08.385812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b05b923-b26a-4787-ba78-dc97123d678a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:08.416865 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17b0f4-9239-431f-bb10-6c2c4b8ade1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:08.417488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17b0f4-9239-431f-bb10-6c2c4b8ade1a admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:08.417771 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17b0f4-9239-431f-bb10-6c2c4b8ade1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:08.418045 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17b0f4-9239-431f-bb10-6c2c4b8ade1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:08.418700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e17b0f4-9239-431f-bb10-6c2c4b8ade1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:08.420629 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e17b0f4-9239-431f-bb10-6c2c4b8ade1a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:08.563990 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1104/2228] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:03:08 2022] POST /identity/v3/projects => generated 383 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:03:08.565099 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d92869-f954-4c3d-ab6e-837d95fb6b21 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:08.575886 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d92869-f954-4c3d-ab6e-837d95fb6b21 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:08.640637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa594b6-cc8b-4860-91b2-ff8290b078fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:08.641193 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa594b6-cc8b-4860-91b2-ff8290b078fe admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:08.641699 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa594b6-cc8b-4860-91b2-ff8290b078fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:08.641921 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa594b6-cc8b-4860-91b2-ff8290b078fe admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:08.642517 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa594b6-cc8b-4860-91b2-ff8290b078fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:08.644505 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa594b6-cc8b-4860-91b2-ff8290b078fe admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:08.724872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1125/2229] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:03:08 2022] POST /identity/v3/users => generated 292 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:03:08.729173 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17b0f4-9239-431f-bb10-6c2c4b8ade1a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:08.743275 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17b0f4-9239-431f-bb10-6c2c4b8ade1a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:08.745220 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16389867-c271-45ac-8629-911dbf3f287e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:08.745844 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16389867-c271-45ac-8629-911dbf3f287e admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:08.746125 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16389867-c271-45ac-8629-911dbf3f287e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:08.746432 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16389867-c271-45ac-8629-911dbf3f287e admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:08.747111 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16389867-c271-45ac-8629-911dbf3f287e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:08.748703 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16389867-c271-45ac-8629-911dbf3f287e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:08.764599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1105/2230] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:03:08.769845 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa594b6-cc8b-4860-91b2-ff8290b078fe admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:08.780186 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa594b6-cc8b-4860-91b2-ff8290b078fe admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:08.785353 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95337c19-3cb5-456d-934d-f6db7ae79b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:08.785967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95337c19-3cb5-456d-934d-f6db7ae79b97 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:08.786245 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95337c19-3cb5-456d-934d-f6db7ae79b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:08.786573 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95337c19-3cb5-456d-934d-f6db7ae79b97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:08.787380 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95337c19-3cb5-456d-934d-f6db7ae79b97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:08.788992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95337c19-3cb5-456d-934d-f6db7ae79b97 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:08.791354 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16389867-c271-45ac-8629-911dbf3f287e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:08.795628 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:08.795852 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-16389867-c271-45ac-8629-911dbf3f287e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:08.801058 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1126/2231] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:03:08.805557 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95337c19-3cb5-456d-934d-f6db7ae79b97 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:08.815102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95337c19-3cb5-456d-934d-f6db7ae79b97 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:08.821070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c492d-68b6-4014-ac83-dc54bae97c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:08.821724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c492d-68b6-4014-ac83-dc54bae97c69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:08.824339 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c492d-68b6-4014-ac83-dc54bae97c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:08.824339 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c492d-68b6-4014-ac83-dc54bae97c69 admin admin] PATH_INFO: `/v3/projects/cd0e5fbbc5484adab345087a118a43d9/users/9361018d89cf4a949fd661478ef3dd02/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:08.841118 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c492d-68b6-4014-ac83-dc54bae97c69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd0e5fbbc5484adab345087a118a43d9, user_id=9361018d89cf4a949fd661478ef3dd02, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:08.843862 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c492d-68b6-4014-ac83-dc54bae97c69 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:08.855028 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:08.860112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16389867-c271-45ac-8629-911dbf3f287e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:08.864049 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaecb05-0c16-4482-922b-6bed13f40333 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:08.864658 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaecb05-0c16-4482-922b-6bed13f40333 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:08.864939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaecb05-0c16-4482-922b-6bed13f40333 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:08.865213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaecb05-0c16-4482-922b-6bed13f40333 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:08.867033 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-afaecb05-0c16-4482-922b-6bed13f40333 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:08.933560 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaecb05-0c16-4482-922b-6bed13f40333 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:08.935052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaecb05-0c16-4482-922b-6bed13f40333 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:08.938610 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1127/2232] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:03:08 2022] PUT /identity/v3/projects/cd0e5fbbc5484adab345087a118a43d9/users/9361018d89cf4a949fd661478ef3dd02/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:03:08.945526 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cbeb33-1481-4ec0-8da2-30e0b29066a4 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:08.945807 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cbeb33-1481-4ec0-8da2-30e0b29066a4 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:08.945984 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1106/2233] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:03:08.946093 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cbeb33-1481-4ec0-8da2-30e0b29066a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:08.974654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a6cbeb33-1481-4ec0-8da2-30e0b29066a4 None None] MFA Rules not processed for user `9361018d89cf4a949fd661478ef3dd02`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:08.975879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a6cbeb33-1481-4ec0-8da2-30e0b29066a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:09.045354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1128/2234] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:03:09.138836 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaecb05-0c16-4482-922b-6bed13f40333 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:09.140769 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-afaecb05-0c16-4482-922b-6bed13f40333 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:09.154839 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaecb05-0c16-4482-922b-6bed13f40333 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:09.208729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba7ae7-2f33-4daf-8793-a610154c44f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:09.211015 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba7ae7-2f33-4daf-8793-a610154c44f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:09.211660 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba7ae7-2f33-4daf-8793-a610154c44f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:09.212079 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba7ae7-2f33-4daf-8793-a610154c44f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:09.226531 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba7ae7-2f33-4daf-8793-a610154c44f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:09.232371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba7ae7-2f33-4daf-8793-a610154c44f6 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:09.240976 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1107/2235] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:03:12.092214 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cbeb33-1481-4ec0-8da2-30e0b29066a4 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:12.094311 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a6cbeb33-1481-4ec0-8da2-30e0b29066a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:12.176079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cbeb33-1481-4ec0-8da2-30e0b29066a4 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:12.181341 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3757c-0a9f-4835-a55f-00f78ce05300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:12.185622 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3757c-0a9f-4835-a55f-00f78ce05300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:12.185907 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3757c-0a9f-4835-a55f-00f78ce05300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:12.186184 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3757c-0a9f-4835-a55f-00f78ce05300 admin admin] PATH_INFO: `/v3/users/69acbf5abd3742daafaf86bac806577f` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:12.187935 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d3757c-0a9f-4835-a55f-00f78ce05300 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69acbf5abd3742daafaf86bac806577f)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:12.189945 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d3757c-0a9f-4835-a55f-00f78ce05300 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:12.401196 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-82d3757c-0a9f-4835-a55f-00f78ce05300 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69acbf5abd3742daafaf86bac806577f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:12.413911 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-82d3757c-0a9f-4835-a55f-00f78ce05300 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69acbf5abd3742daafaf86bac806577f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:12.419887 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:12.421539 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-82d3757c-0a9f-4835-a55f-00f78ce05300 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69acbf5abd3742daafaf86bac806577f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:12.422269 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-82d3757c-0a9f-4835-a55f-00f78ce05300 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69acbf5abd3742daafaf86bac806577f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:12.482299 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-82d3757c-0a9f-4835-a55f-00f78ce05300 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69acbf5abd3742daafaf86bac806577f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:12.489127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1129/2236] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:03:12 2022] DELETE /identity/v3/users/69acbf5abd3742daafaf86bac806577f => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:12.494329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8822a3-9649-4268-9f1e-cfe134639df3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:12.494852 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8822a3-9649-4268-9f1e-cfe134639df3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:12.495116 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8822a3-9649-4268-9f1e-cfe134639df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:12.516547 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-1c8822a3-9649-4268-9f1e-cfe134639df3 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:12.517782 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1c8822a3-9649-4268-9f1e-cfe134639df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:12.579981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1108/2237] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:12.589830 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3757c-0a9f-4835-a55f-00f78ce05300 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:12.591289 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-82d3757c-0a9f-4835-a55f-00f78ce05300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:12.654258 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3757c-0a9f-4835-a55f-00f78ce05300 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:12.657953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4e65c-46d2-4ec7-a424-0e3ac36540ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:12.658806 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4e65c-46d2-4ec7-a424-0e3ac36540ec service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:12.659257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4e65c-46d2-4ec7-a424-0e3ac36540ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:12.662339 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4e65c-46d2-4ec7-a424-0e3ac36540ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:12.669804 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db4e65c-46d2-4ec7-a424-0e3ac36540ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:12.671318 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db4e65c-46d2-4ec7-a424-0e3ac36540ec service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:12.681772 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1130/2238] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:03:13.094369 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8822a3-9649-4268-9f1e-cfe134639df3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:13.095807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-1c8822a3-9649-4268-9f1e-cfe134639df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:13.110106 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8822a3-9649-4268-9f1e-cfe134639df3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:13.115761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b455c-537b-4d4f-af8e-13ed1be02729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:13.116399 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b455c-537b-4d4f-af8e-13ed1be02729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:13.116679 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b455c-537b-4d4f-af8e-13ed1be02729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:13.116968 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b455c-537b-4d4f-af8e-13ed1be02729 admin admin] PATH_INFO: `/v3/projects/e05baef1856b49668ef67ffad4e62e7f` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:13.118131 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b455c-537b-4d4f-af8e-13ed1be02729 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e05baef1856b49668ef67ffad4e62e7f)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:13.120133 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b455c-537b-4d4f-af8e-13ed1be02729 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:13.206486 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:13.216098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b27b455c-537b-4d4f-af8e-13ed1be02729 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e05baef1856b49668ef67ffad4e62e7f', 'request_id': 'req-b27b455c-537b-4d4f-af8e-13ed1be02729'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:13.222286 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b27b455c-537b-4d4f-af8e-13ed1be02729 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e05baef1856b49668ef67ffad4e62e7f', 'request_id': 'req-b27b455c-537b-4d4f-af8e-13ed1be02729'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:13.228414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b27b455c-537b-4d4f-af8e-13ed1be02729 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e05baef1856b49668ef67ffad4e62e7f', 'request_id': 'req-b27b455c-537b-4d4f-af8e-13ed1be02729'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:13.282555 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b27b455c-537b-4d4f-af8e-13ed1be02729 admin admin] The token cache is being invalidate because project e05baef1856b49668ef67ffad4e62e7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:03:13.282866 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b27b455c-537b-4d4f-af8e-13ed1be02729 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:13.283642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b27b455c-537b-4d4f-af8e-13ed1be02729 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:13.285121 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1109/2239] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:03:13 2022] DELETE /identity/v3/projects/e05baef1856b49668ef67ffad4e62e7f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:13.289798 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4e65c-46d2-4ec7-a424-0e3ac36540ec service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:13.300974 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4e65c-46d2-4ec7-a424-0e3ac36540ec service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:13.367615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98286c-74d9-4885-ba98-408b19af4d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:13.368239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98286c-74d9-4885-ba98-408b19af4d6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:13.368509 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98286c-74d9-4885-ba98-408b19af4d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:13.368775 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98286c-74d9-4885-ba98-408b19af4d6b admin admin] PATH_INFO: `/v3/users/c51185609236427bbcb064b766bd2920` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:13.370002 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98286c-74d9-4885-ba98-408b19af4d6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c51185609236427bbcb064b766bd2920)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:13.372019 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98286c-74d9-4885-ba98-408b19af4d6b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:13.823209 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5a98286c-74d9-4885-ba98-408b19af4d6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c51185609236427bbcb064b766bd2920'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:13.837779 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5a98286c-74d9-4885-ba98-408b19af4d6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c51185609236427bbcb064b766bd2920'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:13.851027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5a98286c-74d9-4885-ba98-408b19af4d6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c51185609236427bbcb064b766bd2920'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:13.851887 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5a98286c-74d9-4885-ba98-408b19af4d6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c51185609236427bbcb064b766bd2920'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:13.915709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5a98286c-74d9-4885-ba98-408b19af4d6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c51185609236427bbcb064b766bd2920'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:13.921983 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1131/2240] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:03:13 2022] DELETE /identity/v3/users/c51185609236427bbcb064b766bd2920 => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:03:14.548847 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b455c-537b-4d4f-af8e-13ed1be02729 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:14.552763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b27b455c-537b-4d4f-af8e-13ed1be02729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:14.565420 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b455c-537b-4d4f-af8e-13ed1be02729 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:14.641594 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb826aa-6b00-44a4-8ad8-a0210ce0f052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:14.642286 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb826aa-6b00-44a4-8ad8-a0210ce0f052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:14.642688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb826aa-6b00-44a4-8ad8-a0210ce0f052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:14.642985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb826aa-6b00-44a4-8ad8-a0210ce0f052 admin admin] PATH_INFO: `/v3/projects/df3c2011ab564f0c98a8675e05ec5c7c` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:14.644389 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb826aa-6b00-44a4-8ad8-a0210ce0f052 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df3c2011ab564f0c98a8675e05ec5c7c)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:14.646999 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb826aa-6b00-44a4-8ad8-a0210ce0f052 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:14.758697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0eb826aa-6b00-44a4-8ad8-a0210ce0f052 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df3c2011ab564f0c98a8675e05ec5c7c', 'request_id': 'req-0eb826aa-6b00-44a4-8ad8-a0210ce0f052'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:14.765357 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0eb826aa-6b00-44a4-8ad8-a0210ce0f052 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df3c2011ab564f0c98a8675e05ec5c7c', 'request_id': 'req-0eb826aa-6b00-44a4-8ad8-a0210ce0f052'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:14.771338 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0eb826aa-6b00-44a4-8ad8-a0210ce0f052 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df3c2011ab564f0c98a8675e05ec5c7c', 'request_id': 'req-0eb826aa-6b00-44a4-8ad8-a0210ce0f052'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:14.841481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0eb826aa-6b00-44a4-8ad8-a0210ce0f052 admin admin] The token cache is being invalidate because project df3c2011ab564f0c98a8675e05ec5c7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:03:14.841851 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0eb826aa-6b00-44a4-8ad8-a0210ce0f052 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:14.842536 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-0eb826aa-6b00-44a4-8ad8-a0210ce0f052 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:14.850442 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1110/2241] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:03:14 2022] DELETE /identity/v3/projects/df3c2011ab564f0c98a8675e05ec5c7c => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:03:14.850630 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98286c-74d9-4885-ba98-408b19af4d6b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:14.850630 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5a98286c-74d9-4885-ba98-408b19af4d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:14.859308 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:14.863256 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98286c-74d9-4885-ba98-408b19af4d6b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:14.926612 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de75a3-d5a4-4b12-ae51-23dc20038e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:14.927419 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de75a3-d5a4-4b12-ae51-23dc20038e2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:14.927526 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de75a3-d5a4-4b12-ae51-23dc20038e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:14.927795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de75a3-d5a4-4b12-ae51-23dc20038e2f admin admin] PATH_INFO: `/v3/users/70fe30061654419aa306fdb1453299d3` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:14.929077 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de75a3-d5a4-4b12-ae51-23dc20038e2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=70fe30061654419aa306fdb1453299d3)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:14.939944 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de75a3-d5a4-4b12-ae51-23dc20038e2f admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:15.225690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d3de75a3-d5a4-4b12-ae51-23dc20038e2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70fe30061654419aa306fdb1453299d3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:15.237488 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d3de75a3-d5a4-4b12-ae51-23dc20038e2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70fe30061654419aa306fdb1453299d3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:15.244814 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d3de75a3-d5a4-4b12-ae51-23dc20038e2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70fe30061654419aa306fdb1453299d3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:15.245579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d3de75a3-d5a4-4b12-ae51-23dc20038e2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70fe30061654419aa306fdb1453299d3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:15.312197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d3de75a3-d5a4-4b12-ae51-23dc20038e2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70fe30061654419aa306fdb1453299d3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:15.319709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1132/2242] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:03:14 2022] DELETE /identity/v3/users/70fe30061654419aa306fdb1453299d3 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:15.736116 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb826aa-6b00-44a4-8ad8-a0210ce0f052 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:15.738062 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0eb826aa-6b00-44a4-8ad8-a0210ce0f052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:15.759512 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb826aa-6b00-44a4-8ad8-a0210ce0f052 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:15.835747 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:15.853287 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a7c13-99d9-4e00-b78a-f4d8694cd529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:15.854123 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a7c13-99d9-4e00-b78a-f4d8694cd529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:15.854842 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a7c13-99d9-4e00-b78a-f4d8694cd529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:15.855278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a7c13-99d9-4e00-b78a-f4d8694cd529 admin admin] PATH_INFO: `/v3/projects/565a6a46bb194f979648f44767d67aaf` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:15.856608 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a7c13-99d9-4e00-b78a-f4d8694cd529 admin admin] RBAC: Authorizing `identity:delete_project(project_id=565a6a46bb194f979648f44767d67aaf)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:15.858632 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a7c13-99d9-4e00-b78a-f4d8694cd529 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:15.978616 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-546a7c13-99d9-4e00-b78a-f4d8694cd529 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '565a6a46bb194f979648f44767d67aaf', 'request_id': 'req-546a7c13-99d9-4e00-b78a-f4d8694cd529'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:15.986495 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-546a7c13-99d9-4e00-b78a-f4d8694cd529 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '565a6a46bb194f979648f44767d67aaf', 'request_id': 'req-546a7c13-99d9-4e00-b78a-f4d8694cd529'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:15.994835 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-546a7c13-99d9-4e00-b78a-f4d8694cd529 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '565a6a46bb194f979648f44767d67aaf', 'request_id': 'req-546a7c13-99d9-4e00-b78a-f4d8694cd529'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:16.144503 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-546a7c13-99d9-4e00-b78a-f4d8694cd529 admin admin] The token cache is being invalidate because project 565a6a46bb194f979648f44767d67aaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:03:16.144795 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-546a7c13-99d9-4e00-b78a-f4d8694cd529 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:16.145442 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-546a7c13-99d9-4e00-b78a-f4d8694cd529 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:16.147812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1111/2243] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:03:15 2022] DELETE /identity/v3/projects/565a6a46bb194f979648f44767d67aaf => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:16.152747 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e47d48-d135-4ec8-9e49-aa39b5afd76d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:16.153221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e47d48-d135-4ec8-9e49-aa39b5afd76d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:16.153336 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e47d48-d135-4ec8-9e49-aa39b5afd76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:16.172519 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-f5e47d48-d135-4ec8-9e49-aa39b5afd76d None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:16.175603 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f5e47d48-d135-4ec8-9e49-aa39b5afd76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:16.237829 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1133/2244] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:16.306264 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8cef1-ebb0-40ce-9641-47b94d8a457e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:16.306465 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8cef1-ebb0-40ce-9641-47b94d8a457e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:16.306737 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8cef1-ebb0-40ce-9641-47b94d8a457e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:16.327986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d8a8cef1-ebb0-40ce-9641-47b94d8a457e None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:16.329381 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d8a8cef1-ebb0-40ce-9641-47b94d8a457e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:16.338381 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1112/2245] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:16.342825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e47d48-d135-4ec8-9e49-aa39b5afd76d None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:16.353270 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e47d48-d135-4ec8-9e49-aa39b5afd76d None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:16.357940 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de92f3a4-b29c-453d-9faa-637b7bbd39bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:16.358566 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92f3a4-b29c-453d-9faa-637b7bbd39bc admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:16.358869 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92f3a4-b29c-453d-9faa-637b7bbd39bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:16.359135 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92f3a4-b29c-453d-9faa-637b7bbd39bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:16.359768 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de92f3a4-b29c-453d-9faa-637b7bbd39bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:03:16.360062 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de92f3a4-b29c-453d-9faa-637b7bbd39bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:16.361677 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de92f3a4-b29c-453d-9faa-637b7bbd39bc admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:16.374020 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1134/2246] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:03:16.378536 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8cef1-ebb0-40ce-9641-47b94d8a457e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:16.388735 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8cef1-ebb0-40ce-9641-47b94d8a457e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:16.393407 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb196b-c453-44d1-b134-a0ef1fd2f06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:16.394004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb196b-c453-44d1-b134-a0ef1fd2f06a admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:16.394282 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb196b-c453-44d1-b134-a0ef1fd2f06a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:16.394551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb196b-c453-44d1-b134-a0ef1fd2f06a admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:16.395221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebb196b-c453-44d1-b134-a0ef1fd2f06a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:16.397127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebb196b-c453-44d1-b134-a0ef1fd2f06a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:16.479750 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1113/2247] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:03:16 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:03:16.485795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de92f3a4-b29c-453d-9faa-637b7bbd39bc admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:16.496413 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de92f3a4-b29c-453d-9faa-637b7bbd39bc admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:16.504121 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:16.557764 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8b148-a542-4dd6-bf20-1c1db6341907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:16.558307 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8b148-a542-4dd6-bf20-1c1db6341907 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:16.558579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8b148-a542-4dd6-bf20-1c1db6341907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:16.558865 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8b148-a542-4dd6-bf20-1c1db6341907 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:16.559548 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d8b148-a542-4dd6-bf20-1c1db6341907 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:16.561490 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d8b148-a542-4dd6-bf20-1c1db6341907 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:16.651227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1135/2248] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:03:16 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:03:16.655430 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb196b-c453-44d1-b134-a0ef1fd2f06a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:16.667273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb196b-c453-44d1-b134-a0ef1fd2f06a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:16.672196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb0ed84-bff4-4cea-b329-faaac1da627a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:16.672790 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0ed84-bff4-4cea-b329-faaac1da627a admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:16.673066 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0ed84-bff4-4cea-b329-faaac1da627a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:16.673335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0ed84-bff4-4cea-b329-faaac1da627a admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:16.674029 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb0ed84-bff4-4cea-b329-faaac1da627a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:16.675649 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb0ed84-bff4-4cea-b329-faaac1da627a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:16.687519 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1114/2249] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:03:16.691839 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8b148-a542-4dd6-bf20-1c1db6341907 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:16.702868 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8b148-a542-4dd6-bf20-1c1db6341907 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:16.707625 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6629c-7878-490d-88a7-28518c7e15c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:16.708223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6629c-7878-490d-88a7-28518c7e15c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:16.708495 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6629c-7878-490d-88a7-28518c7e15c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:16.708815 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6629c-7878-490d-88a7-28518c7e15c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:16.709481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6629c-7878-490d-88a7-28518c7e15c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:16.711760 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6629c-7878-490d-88a7-28518c7e15c4 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:16.721429 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1136/2250] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:03:16.725863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb0ed84-bff4-4cea-b329-faaac1da627a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:16.736374 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb0ed84-bff4-4cea-b329-faaac1da627a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:16.740888 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c870164-0768-4002-b8a6-56f18ad44a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:16.741524 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c870164-0768-4002-b8a6-56f18ad44a22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:16.741837 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c870164-0768-4002-b8a6-56f18ad44a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:16.742107 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c870164-0768-4002-b8a6-56f18ad44a22 admin admin] PATH_INFO: `/v3/projects/45f2b88c15b34b18bc9b43009b631f41/users/1e34c80e58264084855769d921415366/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:16.756362 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c870164-0768-4002-b8a6-56f18ad44a22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45f2b88c15b34b18bc9b43009b631f41, user_id=1e34c80e58264084855769d921415366, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:16.759223 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c870164-0768-4002-b8a6-56f18ad44a22 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:16.838572 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1115/2251] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:03:16 2022] PUT /identity/v3/projects/45f2b88c15b34b18bc9b43009b631f41/users/1e34c80e58264084855769d921415366/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:03:16.848827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f72bd4-e839-471a-9c49-c9f0c842ac0f None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:16.852614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f72bd4-e839-471a-9c49-c9f0c842ac0f None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:16.852614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f72bd4-e839-471a-9c49-c9f0c842ac0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:16.879320 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-86f72bd4-e839-471a-9c49-c9f0c842ac0f None None] MFA Rules not processed for user `1e34c80e58264084855769d921415366`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:16.880583 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-86f72bd4-e839-471a-9c49-c9f0c842ac0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:16.954799 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1137/2252] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:16.989185 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c870164-0768-4002-b8a6-56f18ad44a22 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:16.991082 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0c870164-0768-4002-b8a6-56f18ad44a22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:17.082252 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c870164-0768-4002-b8a6-56f18ad44a22 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:17.091124 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d6a6e-5ac8-4c4c-a918-154b8c979496 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:17.092529 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d6a6e-5ac8-4c4c-a918-154b8c979496 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:17.092878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d6a6e-5ac8-4c4c-a918-154b8c979496 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:17.093214 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d6a6e-5ac8-4c4c-a918-154b8c979496 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:17.103466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d6a6e-5ac8-4c4c-a918-154b8c979496 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:17.105076 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d6a6e-5ac8-4c4c-a918-154b8c979496 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:17.114477 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1116/2253] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:03:19.279664 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f72bd4-e839-471a-9c49-c9f0c842ac0f None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:19.281095 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-86f72bd4-e839-471a-9c49-c9f0c842ac0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:19.362403 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f72bd4-e839-471a-9c49-c9f0c842ac0f None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:19.368494 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df64f8d-ca76-4c6a-9008-1dabde6ebf7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:19.371190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df64f8d-ca76-4c6a-9008-1dabde6ebf7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:19.371190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df64f8d-ca76-4c6a-9008-1dabde6ebf7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:19.371190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df64f8d-ca76-4c6a-9008-1dabde6ebf7c admin admin] PATH_INFO: `/v3/users/87f00c68976249498c8378a828c2bc5c` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:19.371190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df64f8d-ca76-4c6a-9008-1dabde6ebf7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=87f00c68976249498c8378a828c2bc5c)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:19.373323 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df64f8d-ca76-4c6a-9008-1dabde6ebf7c admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:19.402574 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:19.605148 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-3df64f8d-ca76-4c6a-9008-1dabde6ebf7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87f00c68976249498c8378a828c2bc5c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:19.626005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-3df64f8d-ca76-4c6a-9008-1dabde6ebf7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87f00c68976249498c8378a828c2bc5c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:19.633727 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-3df64f8d-ca76-4c6a-9008-1dabde6ebf7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87f00c68976249498c8378a828c2bc5c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:19.634783 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-3df64f8d-ca76-4c6a-9008-1dabde6ebf7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87f00c68976249498c8378a828c2bc5c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:19.716757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-3df64f8d-ca76-4c6a-9008-1dabde6ebf7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87f00c68976249498c8378a828c2bc5c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:19.723586 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1138/2254] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:03:19 2022] DELETE /identity/v3/users/87f00c68976249498c8378a828c2bc5c => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:19.728761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b80b8-b942-45f4-bb79-c151609e99aa None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:19.729277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b80b8-b942-45f4-bb79-c151609e99aa None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:19.729372 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b80b8-b942-45f4-bb79-c151609e99aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:19.754744 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-6f6b80b8-b942-45f4-bb79-c151609e99aa None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:19.754744 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6f6b80b8-b942-45f4-bb79-c151609e99aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:19.768697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:19.811261 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1117/2255] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:19.821826 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df64f8d-ca76-4c6a-9008-1dabde6ebf7c admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:19.823475 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3df64f8d-ca76-4c6a-9008-1dabde6ebf7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:19.886013 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df64f8d-ca76-4c6a-9008-1dabde6ebf7c admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:19.889560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5f331-3d5c-4892-9e6e-b20f1481b2a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:19.890166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5f331-3d5c-4892-9e6e-b20f1481b2a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:19.890442 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5f331-3d5c-4892-9e6e-b20f1481b2a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:19.890737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5f331-3d5c-4892-9e6e-b20f1481b2a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:19.900935 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5f331-3d5c-4892-9e6e-b20f1481b2a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:19.902368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5f331-3d5c-4892-9e6e-b20f1481b2a0 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:19.912060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1139/2256] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:03:20.382216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b80b8-b942-45f4-bb79-c151609e99aa None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:20.383645 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6f6b80b8-b942-45f4-bb79-c151609e99aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:20.395865 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b80b8-b942-45f4-bb79-c151609e99aa None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:20.407439 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5f331-3d5c-4892-9e6e-b20f1481b2a0 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:20.409106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-66f5f331-3d5c-4892-9e6e-b20f1481b2a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:20.411983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1faa0-dffa-4154-9ecf-56e50c942ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:20.412763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1faa0-dffa-4154-9ecf-56e50c942ecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:20.412961 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1faa0-dffa-4154-9ecf-56e50c942ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:20.413234 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1faa0-dffa-4154-9ecf-56e50c942ecb admin admin] PATH_INFO: `/v3/users/1e34c80e58264084855769d921415366` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:20.416114 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df1faa0-dffa-4154-9ecf-56e50c942ecb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e34c80e58264084855769d921415366)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:20.419528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df1faa0-dffa-4154-9ecf-56e50c942ecb admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:20.433415 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5f331-3d5c-4892-9e6e-b20f1481b2a0 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:20.438404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa5303-f704-4b70-8e9e-e4a48db1030c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:20.439128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa5303-f704-4b70-8e9e-e4a48db1030c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:20.439407 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa5303-f704-4b70-8e9e-e4a48db1030c admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:20.439732 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa5303-f704-4b70-8e9e-e4a48db1030c admin admin] PATH_INFO: `/v3/projects/1b23618d7cd44d429d5923f62b9c78e6` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:20.440807 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa5303-f704-4b70-8e9e-e4a48db1030c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b23618d7cd44d429d5923f62b9c78e6)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:20.442734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa5303-f704-4b70-8e9e-e4a48db1030c admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:20.602894 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-baaa5303-f704-4b70-8e9e-e4a48db1030c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b23618d7cd44d429d5923f62b9c78e6', 'request_id': 'req-baaa5303-f704-4b70-8e9e-e4a48db1030c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:20.604082 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7df1faa0-dffa-4154-9ecf-56e50c942ecb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e34c80e58264084855769d921415366'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:20.611978 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7df1faa0-dffa-4154-9ecf-56e50c942ecb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e34c80e58264084855769d921415366'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:20.627042 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7df1faa0-dffa-4154-9ecf-56e50c942ecb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e34c80e58264084855769d921415366'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:20.662235 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-baaa5303-f704-4b70-8e9e-e4a48db1030c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b23618d7cd44d429d5923f62b9c78e6', 'request_id': 'req-baaa5303-f704-4b70-8e9e-e4a48db1030c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:20.670623 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-baaa5303-f704-4b70-8e9e-e4a48db1030c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b23618d7cd44d429d5923f62b9c78e6', 'request_id': 'req-baaa5303-f704-4b70-8e9e-e4a48db1030c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:20.683313 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-baaa5303-f704-4b70-8e9e-e4a48db1030c admin admin] The token cache is being invalidate because project 1b23618d7cd44d429d5923f62b9c78e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:03:20.683790 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-baaa5303-f704-4b70-8e9e-e4a48db1030c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:20.684614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-baaa5303-f704-4b70-8e9e-e4a48db1030c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:20.686288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1140/2257] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:03:20 2022] DELETE /identity/v3/projects/1b23618d7cd44d429d5923f62b9c78e6 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:03:20.690679 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa5303-f704-4b70-8e9e-e4a48db1030c admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:20.692491 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-baaa5303-f704-4b70-8e9e-e4a48db1030c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:20.735518 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:20.766983 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7df1faa0-dffa-4154-9ecf-56e50c942ecb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e34c80e58264084855769d921415366'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:20.768050 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7df1faa0-dffa-4154-9ecf-56e50c942ecb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e34c80e58264084855769d921415366'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:20.787081 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1118/2258] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:03:20 2022] DELETE /identity/v3/users/1e34c80e58264084855769d921415366 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:20.789337 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb9290-d5b1-4d1c-a018-b5c2b0c36e28 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:20.792750 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb9290-d5b1-4d1c-a018-b5c2b0c36e28 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:20.793202 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb9290-d5b1-4d1c-a018-b5c2b0c36e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:20.803624 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa5303-f704-4b70-8e9e-e4a48db1030c admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:20.820938 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b5bb9290-d5b1-4d1c-a018-b5c2b0c36e28 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:20.822444 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b5bb9290-d5b1-4d1c-a018-b5c2b0c36e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:20.931044 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024cd24c-2f4e-45c2-b5bd-9cc98fe1eceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:20.932254 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024cd24c-2f4e-45c2-b5bd-9cc98fe1eceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:20.932386 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024cd24c-2f4e-45c2-b5bd-9cc98fe1eceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:20.932796 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024cd24c-2f4e-45c2-b5bd-9cc98fe1eceb admin admin] PATH_INFO: `/v3/users/559ef77d2bce459fb75f80e4871e2434` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:20.934625 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024cd24c-2f4e-45c2-b5bd-9cc98fe1eceb admin admin] RBAC: Authorizing `identity:delete_user(user_id=559ef77d2bce459fb75f80e4871e2434)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:20.935763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1119/2259] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:20.941158 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024cd24c-2f4e-45c2-b5bd-9cc98fe1eceb admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:20.949238 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb9290-d5b1-4d1c-a018-b5c2b0c36e28 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:20.951011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b5bb9290-d5b1-4d1c-a018-b5c2b0c36e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:20.996764 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:21.026427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb9290-d5b1-4d1c-a018-b5c2b0c36e28 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:21.026644 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f9c5e-6f9e-48a9-a371-340157f13c9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:21.027139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f9c5e-6f9e-48a9-a371-340157f13c9a service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:21.027415 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f9c5e-6f9e-48a9-a371-340157f13c9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:21.027682 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f9c5e-6f9e-48a9-a371-340157f13c9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:21.037928 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f9c5e-6f9e-48a9-a371-340157f13c9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:21.040113 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f9c5e-6f9e-48a9-a371-340157f13c9a service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:21.051553 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1120/2260] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:03:21.283181 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-024cd24c-2f4e-45c2-b5bd-9cc98fe1eceb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '559ef77d2bce459fb75f80e4871e2434'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:21.308792 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-024cd24c-2f4e-45c2-b5bd-9cc98fe1eceb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '559ef77d2bce459fb75f80e4871e2434'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:21.321175 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-024cd24c-2f4e-45c2-b5bd-9cc98fe1eceb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '559ef77d2bce459fb75f80e4871e2434'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:21.321806 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-024cd24c-2f4e-45c2-b5bd-9cc98fe1eceb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '559ef77d2bce459fb75f80e4871e2434'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:21.418276 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-024cd24c-2f4e-45c2-b5bd-9cc98fe1eceb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '559ef77d2bce459fb75f80e4871e2434'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:21.423567 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f9c5e-6f9e-48a9-a371-340157f13c9a service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:21.425802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c38f9c5e-6f9e-48a9-a371-340157f13c9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:21.430907 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1141/2261] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:03:20 2022] DELETE /identity/v3/users/559ef77d2bce459fb75f80e4871e2434 => generated 0 bytes in 741 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:21.447137 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f9c5e-6f9e-48a9-a371-340157f13c9a service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:21.543707 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ce68f-044e-4120-813d-5f290125240d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:21.544654 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ce68f-044e-4120-813d-5f290125240d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:21.545095 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ce68f-044e-4120-813d-5f290125240d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:21.545511 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ce68f-044e-4120-813d-5f290125240d admin admin] PATH_INFO: `/v3/users/9361018d89cf4a949fd661478ef3dd02` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:21.547496 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ce68f-044e-4120-813d-5f290125240d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9361018d89cf4a949fd661478ef3dd02)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:21.550267 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ce68f-044e-4120-813d-5f290125240d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:21.675605 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024cd24c-2f4e-45c2-b5bd-9cc98fe1eceb admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:21.675605 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-024cd24c-2f4e-45c2-b5bd-9cc98fe1eceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:21.686429 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024cd24c-2f4e-45c2-b5bd-9cc98fe1eceb admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:21.704529 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31e394-2b9a-4738-8076-4293c094f76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:21.705246 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31e394-2b9a-4738-8076-4293c094f76b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:21.705524 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31e394-2b9a-4738-8076-4293c094f76b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:21.705918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31e394-2b9a-4738-8076-4293c094f76b admin admin] PATH_INFO: `/v3/projects/45f2b88c15b34b18bc9b43009b631f41` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:21.707056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be31e394-2b9a-4738-8076-4293c094f76b admin admin] RBAC: Authorizing `identity:delete_project(project_id=45f2b88c15b34b18bc9b43009b631f41)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:21.717195 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be31e394-2b9a-4738-8076-4293c094f76b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:21.816672 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9b0ce68f-044e-4120-813d-5f290125240d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9361018d89cf4a949fd661478ef3dd02'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:21.828699 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9b0ce68f-044e-4120-813d-5f290125240d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9361018d89cf4a949fd661478ef3dd02'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:21.845301 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9b0ce68f-044e-4120-813d-5f290125240d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9361018d89cf4a949fd661478ef3dd02'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:21.865983 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:21.869889 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-be31e394-2b9a-4738-8076-4293c094f76b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45f2b88c15b34b18bc9b43009b631f41', 'request_id': 'req-be31e394-2b9a-4738-8076-4293c094f76b'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:21.956324 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9b0ce68f-044e-4120-813d-5f290125240d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9361018d89cf4a949fd661478ef3dd02'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:21.956887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9b0ce68f-044e-4120-813d-5f290125240d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9361018d89cf4a949fd661478ef3dd02'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:21.966073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1121/2262] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:03:21 2022] DELETE /identity/v3/users/9361018d89cf4a949fd661478ef3dd02 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:21.975180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa564ef-5402-4e59-94a6-f1253b1b4a33 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:21.975662 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa564ef-5402-4e59-94a6-f1253b1b4a33 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:21.976803 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa564ef-5402-4e59-94a6-f1253b1b4a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:22.014306 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5aa564ef-5402-4e59-94a6-f1253b1b4a33 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:22.015961 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5aa564ef-5402-4e59-94a6-f1253b1b4a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:22.082242 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-be31e394-2b9a-4738-8076-4293c094f76b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45f2b88c15b34b18bc9b43009b631f41', 'request_id': 'req-be31e394-2b9a-4738-8076-4293c094f76b'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:22.109137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-be31e394-2b9a-4738-8076-4293c094f76b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45f2b88c15b34b18bc9b43009b631f41', 'request_id': 'req-be31e394-2b9a-4738-8076-4293c094f76b'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:22.113218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:22.128030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1122/2263] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:22.139217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa564ef-5402-4e59-94a6-f1253b1b4a33 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:22.141046 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5aa564ef-5402-4e59-94a6-f1253b1b4a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:22.142464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-be31e394-2b9a-4738-8076-4293c094f76b admin admin] The token cache is being invalidate because project 45f2b88c15b34b18bc9b43009b631f41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:03:22.142737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-be31e394-2b9a-4738-8076-4293c094f76b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:22.143378 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-be31e394-2b9a-4738-8076-4293c094f76b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:22.144855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1142/2264] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:03:21 2022] DELETE /identity/v3/projects/45f2b88c15b34b18bc9b43009b631f41 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:22.185291 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31e394-2b9a-4738-8076-4293c094f76b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:22.189275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-be31e394-2b9a-4738-8076-4293c094f76b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:22.251985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa564ef-5402-4e59-94a6-f1253b1b4a33 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:22.261198 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd769b-8f66-4f83-95c2-24e62fa19d00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:22.261850 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd769b-8f66-4f83-95c2-24e62fa19d00 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:22.262129 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd769b-8f66-4f83-95c2-24e62fa19d00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:22.262400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd769b-8f66-4f83-95c2-24e62fa19d00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:22.267405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b1fd769b-8f66-4f83-95c2-24e62fa19d00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:22.283561 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd769b-8f66-4f83-95c2-24e62fa19d00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:22.285162 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd769b-8f66-4f83-95c2-24e62fa19d00 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:22.304947 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1123/2265] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:03:22.329107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31e394-2b9a-4738-8076-4293c094f76b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:22.334263 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff0e03-78bb-4cf8-86fb-041828bdc914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:22.334997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff0e03-78bb-4cf8-86fb-041828bdc914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:22.336187 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff0e03-78bb-4cf8-86fb-041828bdc914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:22.336373 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff0e03-78bb-4cf8-86fb-041828bdc914 admin admin] PATH_INFO: `/v3/projects/610265409c564f7aac67d31edd6152f5` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:22.339530 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aff0e03-78bb-4cf8-86fb-041828bdc914 admin admin] RBAC: Authorizing `identity:delete_project(project_id=610265409c564f7aac67d31edd6152f5)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:22.340987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aff0e03-78bb-4cf8-86fb-041828bdc914 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:22.505487 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8aff0e03-78bb-4cf8-86fb-041828bdc914 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '610265409c564f7aac67d31edd6152f5', 'request_id': 'req-8aff0e03-78bb-4cf8-86fb-041828bdc914'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:22.647427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8aff0e03-78bb-4cf8-86fb-041828bdc914 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '610265409c564f7aac67d31edd6152f5', 'request_id': 'req-8aff0e03-78bb-4cf8-86fb-041828bdc914'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:22.670343 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8aff0e03-78bb-4cf8-86fb-041828bdc914 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '610265409c564f7aac67d31edd6152f5', 'request_id': 'req-8aff0e03-78bb-4cf8-86fb-041828bdc914'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:22.690471 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8aff0e03-78bb-4cf8-86fb-041828bdc914 admin admin] The token cache is being invalidate because project 610265409c564f7aac67d31edd6152f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:03:22.690780 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8aff0e03-78bb-4cf8-86fb-041828bdc914 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:22.691482 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8aff0e03-78bb-4cf8-86fb-041828bdc914 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:22.694006 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1143/2266] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:03:22 2022] DELETE /identity/v3/projects/610265409c564f7aac67d31edd6152f5 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:03:22.698479 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd769b-8f66-4f83-95c2-24e62fa19d00 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:22.713040 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd769b-8f66-4f83-95c2-24e62fa19d00 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:22.793073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf3b5e1-d066-4516-b04e-7875dfc814a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:22.793691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf3b5e1-d066-4516-b04e-7875dfc814a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:22.793973 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf3b5e1-d066-4516-b04e-7875dfc814a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:22.794249 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf3b5e1-d066-4516-b04e-7875dfc814a6 admin admin] PATH_INFO: `/v3/users/464947ee8e2746b8a5f48333f36fa007` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:22.795433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf3b5e1-d066-4516-b04e-7875dfc814a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=464947ee8e2746b8a5f48333f36fa007)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:22.797742 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf3b5e1-d066-4516-b04e-7875dfc814a6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:22.894267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff0e03-78bb-4cf8-86fb-041828bdc914 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:22.895838 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8aff0e03-78bb-4cf8-86fb-041828bdc914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:22.913650 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff0e03-78bb-4cf8-86fb-041828bdc914 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:22.918946 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8d336-bcc4-4e25-b1f2-8a8a88cb7bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:22.919733 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8d336-bcc4-4e25-b1f2-8a8a88cb7bc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:22.920218 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8d336-bcc4-4e25-b1f2-8a8a88cb7bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:22.920694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8d336-bcc4-4e25-b1f2-8a8a88cb7bc2 admin admin] PATH_INFO: `/v3/projects/cd0e5fbbc5484adab345087a118a43d9` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:22.922650 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff8d336-bcc4-4e25-b1f2-8a8a88cb7bc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd0e5fbbc5484adab345087a118a43d9)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:22.927066 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff8d336-bcc4-4e25-b1f2-8a8a88cb7bc2 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:23.026718 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fdf3b5e1-d066-4516-b04e-7875dfc814a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '464947ee8e2746b8a5f48333f36fa007'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:23.033690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fdf3b5e1-d066-4516-b04e-7875dfc814a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '464947ee8e2746b8a5f48333f36fa007'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:23.046722 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fdf3b5e1-d066-4516-b04e-7875dfc814a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '464947ee8e2746b8a5f48333f36fa007'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:23.082798 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5ff8d336-bcc4-4e25-b1f2-8a8a88cb7bc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd0e5fbbc5484adab345087a118a43d9', 'request_id': 'req-5ff8d336-bcc4-4e25-b1f2-8a8a88cb7bc2'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:23.134692 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fdf3b5e1-d066-4516-b04e-7875dfc814a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '464947ee8e2746b8a5f48333f36fa007'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:23.135407 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-fdf3b5e1-d066-4516-b04e-7875dfc814a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '464947ee8e2746b8a5f48333f36fa007'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:23.142342 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1124/2267] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:03:22 2022] DELETE /identity/v3/users/464947ee8e2746b8a5f48333f36fa007 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:03:23.198008 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5ff8d336-bcc4-4e25-b1f2-8a8a88cb7bc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd0e5fbbc5484adab345087a118a43d9', 'request_id': 'req-5ff8d336-bcc4-4e25-b1f2-8a8a88cb7bc2'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:23.202097 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:DELETE=17,keystone:SELECT=80,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:23.209219 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5ff8d336-bcc4-4e25-b1f2-8a8a88cb7bc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd0e5fbbc5484adab345087a118a43d9', 'request_id': 'req-5ff8d336-bcc4-4e25-b1f2-8a8a88cb7bc2'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:23.221691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5ff8d336-bcc4-4e25-b1f2-8a8a88cb7bc2 admin admin] The token cache is being invalidate because project cd0e5fbbc5484adab345087a118a43d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:03:23.222083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5ff8d336-bcc4-4e25-b1f2-8a8a88cb7bc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:23.223302 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5ff8d336-bcc4-4e25-b1f2-8a8a88cb7bc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:23.224455 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1144/2268] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:03:22 2022] DELETE /identity/v3/projects/cd0e5fbbc5484adab345087a118a43d9 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:03:23.230708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc657c-55b8-4b41-82ae-f22fbf8325e7 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:23.230992 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc657c-55b8-4b41-82ae-f22fbf8325e7 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:23.231813 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc657c-55b8-4b41-82ae-f22fbf8325e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:23.256026 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-04fc657c-55b8-4b41-82ae-f22fbf8325e7 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:23.257897 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-04fc657c-55b8-4b41-82ae-f22fbf8325e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:23.271571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:23.328517 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1125/2269] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:03:23.336993 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cac771-4597-4f1b-b5b9-b43ea90ce8d8 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:23.337281 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cac771-4597-4f1b-b5b9-b43ea90ce8d8 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:23.337555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cac771-4597-4f1b-b5b9-b43ea90ce8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:23.358843 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-96cac771-4597-4f1b-b5b9-b43ea90ce8d8 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:23.360045 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-96cac771-4597-4f1b-b5b9-b43ea90ce8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:23.368704 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1145/2270] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:03:23.373332 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc657c-55b8-4b41-82ae-f22fbf8325e7 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:23.384924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc657c-55b8-4b41-82ae-f22fbf8325e7 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:23.390033 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104b137-dd2a-48d4-a0d0-64d9e39711cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:23.390840 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104b137-dd2a-48d4-a0d0-64d9e39711cd admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:23.391139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104b137-dd2a-48d4-a0d0-64d9e39711cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:23.391385 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104b137-dd2a-48d4-a0d0-64d9e39711cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:23.392041 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104b137-dd2a-48d4-a0d0-64d9e39711cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:03:23.392339 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104b137-dd2a-48d4-a0d0-64d9e39711cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:23.399105 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104b137-dd2a-48d4-a0d0-64d9e39711cd admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:23.406847 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1126/2271] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:03:23.411966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cac771-4597-4f1b-b5b9-b43ea90ce8d8 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:23.422035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cac771-4597-4f1b-b5b9-b43ea90ce8d8 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:23.426797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375bc20d-c565-4b43-b131-2b7bf5a356d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:23.427405 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bc20d-c565-4b43-b131-2b7bf5a356d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:23.427724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bc20d-c565-4b43-b131-2b7bf5a356d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:23.427989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bc20d-c565-4b43-b131-2b7bf5a356d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:23.428615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375bc20d-c565-4b43-b131-2b7bf5a356d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:23.431170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375bc20d-c565-4b43-b131-2b7bf5a356d0 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:23.542918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1146/2272] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:03:23 2022] POST /identity/v3/projects => generated 381 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:03:23.547467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104b137-dd2a-48d4-a0d0-64d9e39711cd admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:23.560316 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104b137-dd2a-48d4-a0d0-64d9e39711cd admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:23.629815 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912b116-4a43-43f1-9898-f6f8aed00151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:23.630417 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912b116-4a43-43f1-9898-f6f8aed00151 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:23.637000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912b116-4a43-43f1-9898-f6f8aed00151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:23.637000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912b116-4a43-43f1-9898-f6f8aed00151 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:23.637000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912b116-4a43-43f1-9898-f6f8aed00151 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:23.637000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912b116-4a43-43f1-9898-f6f8aed00151 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:23.732478 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1127/2273] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:03:23 2022] POST /identity/v3/users => generated 291 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:03:23.733808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375bc20d-c565-4b43-b131-2b7bf5a356d0 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:23.745768 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375bc20d-c565-4b43-b131-2b7bf5a356d0 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:23.751476 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92603fa2-eb0b-4b72-b0a8-63b9c2886f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:23.751712 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92603fa2-eb0b-4b72-b0a8-63b9c2886f67 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:23.751901 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92603fa2-eb0b-4b72-b0a8-63b9c2886f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:23.752724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92603fa2-eb0b-4b72-b0a8-63b9c2886f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:23.753661 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92603fa2-eb0b-4b72-b0a8-63b9c2886f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:23.755583 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92603fa2-eb0b-4b72-b0a8-63b9c2886f67 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:23.766426 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1147/2274] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:03:23.770727 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912b116-4a43-43f1-9898-f6f8aed00151 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:23.782039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912b116-4a43-43f1-9898-f6f8aed00151 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:23.787074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d2731-6837-44ff-81d3-8b0c5815034f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:23.787678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d2731-6837-44ff-81d3-8b0c5815034f admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:23.787951 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d2731-6837-44ff-81d3-8b0c5815034f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:23.788803 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d2731-6837-44ff-81d3-8b0c5815034f admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:23.789405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d2731-6837-44ff-81d3-8b0c5815034f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:23.791174 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d2731-6837-44ff-81d3-8b0c5815034f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:23.802277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1128/2275] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:03:23.808131 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92603fa2-eb0b-4b72-b0a8-63b9c2886f67 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:23.820435 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92603fa2-eb0b-4b72-b0a8-63b9c2886f67 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:23.826107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee3efc-563d-4fb2-9c1f-40b55d774b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:23.827603 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee3efc-563d-4fb2-9c1f-40b55d774b7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:23.827882 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee3efc-563d-4fb2-9c1f-40b55d774b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:23.828149 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee3efc-563d-4fb2-9c1f-40b55d774b7b admin admin] PATH_INFO: `/v3/projects/fa03f6136eb640ad803659779ce1b085/users/017684196c8e49e0894d38daf83bfaeb/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:23.843801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee3efc-563d-4fb2-9c1f-40b55d774b7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa03f6136eb640ad803659779ce1b085, user_id=017684196c8e49e0894d38daf83bfaeb, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:23.846443 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee3efc-563d-4fb2-9c1f-40b55d774b7b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:23.871930 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d2731-6837-44ff-81d3-8b0c5815034f admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:23.873783 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f26d2731-6837-44ff-81d3-8b0c5815034f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:23.889067 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d2731-6837-44ff-81d3-8b0c5815034f admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:23.895486 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d7ad2-479a-498a-bc74-72634a5661cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:23.895829 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d7ad2-479a-498a-bc74-72634a5661cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:23.896106 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d7ad2-479a-498a-bc74-72634a5661cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:23.896375 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d7ad2-479a-498a-bc74-72634a5661cf admin admin] PATH_INFO: `/v3/projects/06bb3edd4bcf4961b06317fe3e509209` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:23.897479 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5d7ad2-479a-498a-bc74-72634a5661cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=06bb3edd4bcf4961b06317fe3e509209)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:23.900383 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5d7ad2-479a-498a-bc74-72634a5661cf admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:23.952608 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1148/2276] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:03:23 2022] PUT /identity/v3/projects/fa03f6136eb640ad803659779ce1b085/users/017684196c8e49e0894d38daf83bfaeb/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:03:23.957490 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d3f6d-a6c2-4d72-b5cf-38b7ee3eb28f None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:23.957834 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d3f6d-a6c2-4d72-b5cf-38b7ee3eb28f None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:23.958108 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d3f6d-a6c2-4d72-b5cf-38b7ee3eb28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:23.978060 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-db7d3f6d-a6c2-4d72-b5cf-38b7ee3eb28f None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:23.979320 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-db7d3f6d-a6c2-4d72-b5cf-38b7ee3eb28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:24.017649 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dd5d7ad2-479a-498a-bc74-72634a5661cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06bb3edd4bcf4961b06317fe3e509209', 'request_id': 'req-dd5d7ad2-479a-498a-bc74-72634a5661cf'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:24.024750 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dd5d7ad2-479a-498a-bc74-72634a5661cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06bb3edd4bcf4961b06317fe3e509209', 'request_id': 'req-dd5d7ad2-479a-498a-bc74-72634a5661cf'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:24.030595 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dd5d7ad2-479a-498a-bc74-72634a5661cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06bb3edd4bcf4961b06317fe3e509209', 'request_id': 'req-dd5d7ad2-479a-498a-bc74-72634a5661cf'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:24.039622 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1149/2277] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:03:24.049266 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d3f6d-a6c2-4d72-b5cf-38b7ee3eb28f None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:24.050695 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-db7d3f6d-a6c2-4d72-b5cf-38b7ee3eb28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:24.108455 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d3f6d-a6c2-4d72-b5cf-38b7ee3eb28f None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:24.132141 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dd5d7ad2-479a-498a-bc74-72634a5661cf admin admin] The token cache is being invalidate because project 06bb3edd4bcf4961b06317fe3e509209 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:03:24.132459 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dd5d7ad2-479a-498a-bc74-72634a5661cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:24.133101 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dd5d7ad2-479a-498a-bc74-72634a5661cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:24.138101 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1129/2278] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:03:23 2022] DELETE /identity/v3/projects/06bb3edd4bcf4961b06317fe3e509209 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:03:24.139993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdcbb4-e01b-467e-a971-cffc17825859 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:24.140280 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdcbb4-e01b-467e-a971-cffc17825859 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:24.140556 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdcbb4-e01b-467e-a971-cffc17825859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:24.146783 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:24.158462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e229c7-24b1-4928-b8a0-3841faaa61dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:24.159279 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e229c7-24b1-4928-b8a0-3841faaa61dc service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:24.159621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e229c7-24b1-4928-b8a0-3841faaa61dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:24.159819 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e229c7-24b1-4928-b8a0-3841faaa61dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:24.160502 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f3bdcbb4-e01b-467e-a971-cffc17825859 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:24.161638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-17e229c7-24b1-4928-b8a0-3841faaa61dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:24.161778 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f3bdcbb4-e01b-467e-a971-cffc17825859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:24.166807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:24.223543 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1130/2279] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:03:24.225187 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e229c7-24b1-4928-b8a0-3841faaa61dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:24.227011 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e229c7-24b1-4928-b8a0-3841faaa61dc service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:24.231251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454209b-d493-4151-ad10-6cd0b77970cb None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:24.231531 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454209b-d493-4151-ad10-6cd0b77970cb None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:24.232473 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454209b-d493-4151-ad10-6cd0b77970cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:24.236927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1150/2280] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:03:24.252412 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-8454209b-d493-4151-ad10-6cd0b77970cb None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:24.257608 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8454209b-d493-4151-ad10-6cd0b77970cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:24.262019 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1131/2281] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:03:24.266361 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e229c7-24b1-4928-b8a0-3841faaa61dc service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:24.277090 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e229c7-24b1-4928-b8a0-3841faaa61dc service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:24.281723 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66a25f-881e-49b6-ab9e-61c107103a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:24.282306 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66a25f-881e-49b6-ab9e-61c107103a82 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:24.282580 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66a25f-881e-49b6-ab9e-61c107103a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:24.282872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66a25f-881e-49b6-ab9e-61c107103a82 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:24.283511 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca66a25f-881e-49b6-ab9e-61c107103a82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:03:24.283805 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca66a25f-881e-49b6-ab9e-61c107103a82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:24.285374 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca66a25f-881e-49b6-ab9e-61c107103a82 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:24.297086 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1151/2282] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:03:24.301346 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454209b-d493-4151-ad10-6cd0b77970cb None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:24.311323 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454209b-d493-4151-ad10-6cd0b77970cb None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:24.316029 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a1dd9-1602-4971-9633-b0227d21f8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:24.316630 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a1dd9-1602-4971-9633-b0227d21f8af admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:24.316905 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a1dd9-1602-4971-9633-b0227d21f8af admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:24.317174 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a1dd9-1602-4971-9633-b0227d21f8af admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:24.317802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a1dd9-1602-4971-9633-b0227d21f8af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:24.320711 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a1dd9-1602-4971-9633-b0227d21f8af admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:24.437691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1132/2283] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:03:24 2022] POST /identity/v3/projects => generated 383 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:03:24.440360 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66a25f-881e-49b6-ab9e-61c107103a82 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:24.449669 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66a25f-881e-49b6-ab9e-61c107103a82 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:24.509947 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae24de-f463-42a3-bb61-db2639927eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:24.510553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae24de-f463-42a3-bb61-db2639927eeb admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:24.510857 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae24de-f463-42a3-bb61-db2639927eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:24.511129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae24de-f463-42a3-bb61-db2639927eeb admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:24.511745 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae24de-f463-42a3-bb61-db2639927eeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:24.513677 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae24de-f463-42a3-bb61-db2639927eeb admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:24.641693 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1152/2284] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:03:24 2022] POST /identity/v3/users => generated 292 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:03:24.645834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a1dd9-1602-4971-9633-b0227d21f8af admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:24.661810 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a1dd9-1602-4971-9633-b0227d21f8af admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:24.667091 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd22bb-0fdf-4669-b5d9-6278c35b2a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:24.667706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd22bb-0fdf-4669-b5d9-6278c35b2a76 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:24.667985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd22bb-0fdf-4669-b5d9-6278c35b2a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:24.668262 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd22bb-0fdf-4669-b5d9-6278c35b2a76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:24.668909 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fd22bb-0fdf-4669-b5d9-6278c35b2a76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:24.670521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fd22bb-0fdf-4669-b5d9-6278c35b2a76 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:24.681432 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1133/2285] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:03:24.685298 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae24de-f463-42a3-bb61-db2639927eeb admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:24.697184 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae24de-f463-42a3-bb61-db2639927eeb admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:24.702039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c66bee-60d4-4a58-ac10-c529cfa0b42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:24.705230 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c66bee-60d4-4a58-ac10-c529cfa0b42c admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:24.705230 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c66bee-60d4-4a58-ac10-c529cfa0b42c admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:24.705230 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c66bee-60d4-4a58-ac10-c529cfa0b42c admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:24.705230 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c66bee-60d4-4a58-ac10-c529cfa0b42c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:24.705590 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c66bee-60d4-4a58-ac10-c529cfa0b42c admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:24.716294 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1153/2286] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:03:24.720396 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd22bb-0fdf-4669-b5d9-6278c35b2a76 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:24.732118 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd22bb-0fdf-4669-b5d9-6278c35b2a76 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:24.736640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75da3e-7b24-40df-bc11-8b13a98f7f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:24.737332 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75da3e-7b24-40df-bc11-8b13a98f7f12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:24.737553 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75da3e-7b24-40df-bc11-8b13a98f7f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:24.737822 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75da3e-7b24-40df-bc11-8b13a98f7f12 admin admin] PATH_INFO: `/v3/projects/026452f1f397459fb8f7cf463a9146f6/users/27036d13b9fb49798b8e5606bc422181/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:24.752742 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e75da3e-7b24-40df-bc11-8b13a98f7f12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=026452f1f397459fb8f7cf463a9146f6, user_id=27036d13b9fb49798b8e5606bc422181, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:24.755430 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e75da3e-7b24-40df-bc11-8b13a98f7f12 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:24.854917 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1134/2287] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:03:24 2022] PUT /identity/v3/projects/026452f1f397459fb8f7cf463a9146f6/users/27036d13b9fb49798b8e5606bc422181/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:24.866167 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb630460-ae9c-434e-a973-b74ebccc207d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:24.866533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb630460-ae9c-434e-a973-b74ebccc207d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:24.866912 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb630460-ae9c-434e-a973-b74ebccc207d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:24.897130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-bb630460-ae9c-434e-a973-b74ebccc207d None None] MFA Rules not processed for user `27036d13b9fb49798b8e5606bc422181`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:24.898892 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-bb630460-ae9c-434e-a973-b74ebccc207d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:24.971644 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:25.001063 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1154/2288] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:25.005698 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75da3e-7b24-40df-bc11-8b13a98f7f12 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:25.083863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75da3e-7b24-40df-bc11-8b13a98f7f12 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:25.088762 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826acd0-e22d-478d-9df0-e1a7f2558983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:25.089369 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826acd0-e22d-478d-9df0-e1a7f2558983 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:25.089651 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826acd0-e22d-478d-9df0-e1a7f2558983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:25.089925 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826acd0-e22d-478d-9df0-e1a7f2558983 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:25.090591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826acd0-e22d-478d-9df0-e1a7f2558983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:25.092730 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826acd0-e22d-478d-9df0-e1a7f2558983 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:25.218082 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1135/2289] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:03:25 2022] POST /identity/v3/projects => generated 385 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:03:25.221898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb630460-ae9c-434e-a973-b74ebccc207d None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:25.233614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb630460-ae9c-434e-a973-b74ebccc207d None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:25.296170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6a70e-003d-4e68-9875-bec874cd20e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:25.296715 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6a70e-003d-4e68-9875-bec874cd20e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:25.296985 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6a70e-003d-4e68-9875-bec874cd20e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:25.297303 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6a70e-003d-4e68-9875-bec874cd20e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:25.297934 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd6a70e-003d-4e68-9875-bec874cd20e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:25.300346 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd6a70e-003d-4e68-9875-bec874cd20e3 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:25.404306 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1155/2290] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:03:25 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:03:25.408338 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826acd0-e22d-478d-9df0-e1a7f2558983 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:25.418708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826acd0-e22d-478d-9df0-e1a7f2558983 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:25.423777 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668c144-3b82-4c58-a692-d21166756745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:25.424369 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668c144-3b82-4c58-a692-d21166756745 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:25.424646 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668c144-3b82-4c58-a692-d21166756745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:25.424920 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668c144-3b82-4c58-a692-d21166756745 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:25.425620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668c144-3b82-4c58-a692-d21166756745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:25.427269 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668c144-3b82-4c58-a692-d21166756745 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:25.438259 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1136/2291] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:03:25.442106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6a70e-003d-4e68-9875-bec874cd20e3 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:25.452768 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6a70e-003d-4e68-9875-bec874cd20e3 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:25.457287 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed619255-5d6b-43f1-85fa-d54f5c0fb33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:25.457869 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed619255-5d6b-43f1-85fa-d54f5c0fb33a admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:25.458143 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed619255-5d6b-43f1-85fa-d54f5c0fb33a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:25.458410 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed619255-5d6b-43f1-85fa-d54f5c0fb33a admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:25.459100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed619255-5d6b-43f1-85fa-d54f5c0fb33a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:25.460864 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed619255-5d6b-43f1-85fa-d54f5c0fb33a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:25.470673 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1156/2292] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:03:25.474868 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668c144-3b82-4c58-a692-d21166756745 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:25.484841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668c144-3b82-4c58-a692-d21166756745 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:25.489784 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9639a27-696f-4383-950a-f850f0181dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:25.490456 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9639a27-696f-4383-950a-f850f0181dd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:25.490770 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9639a27-696f-4383-950a-f850f0181dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:25.491044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9639a27-696f-4383-950a-f850f0181dd8 admin admin] PATH_INFO: `/v3/projects/32d405ad662a43e1924b6bd3448f014d/users/fa796f6f4b6643738fddaf2d0c1651d8/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:25.508452 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9639a27-696f-4383-950a-f850f0181dd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32d405ad662a43e1924b6bd3448f014d, user_id=fa796f6f4b6643738fddaf2d0c1651d8, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:25.511438 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9639a27-696f-4383-950a-f850f0181dd8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:25.514397 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:25.609966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1137/2293] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:03:25 2022] PUT /identity/v3/projects/32d405ad662a43e1924b6bd3448f014d/users/fa796f6f4b6643738fddaf2d0c1651d8/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:25.620729 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852e9e5-91a0-4f99-ab38-f6e7f94aa2f8 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:25.621015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852e9e5-91a0-4f99-ab38-f6e7f94aa2f8 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:25.621292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852e9e5-91a0-4f99-ab38-f6e7f94aa2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:25.673944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252619ca-b2d4-4c50-b800-cdd6717f7323 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:25.674224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252619ca-b2d4-4c50-b800-cdd6717f7323 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:25.674506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252619ca-b2d4-4c50-b800-cdd6717f7323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:25.681105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-6852e9e5-91a0-4f99-ab38-f6e7f94aa2f8 None None] MFA Rules not processed for user `fa796f6f4b6643738fddaf2d0c1651d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:25.688868 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6852e9e5-91a0-4f99-ab38-f6e7f94aa2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:25.786125 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1157/2294] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:03:25.792304 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852e9e5-91a0-4f99-ab38-f6e7f94aa2f8 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:25.853797 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-252619ca-b2d4-4c50-b800-cdd6717f7323 None None] MFA Rules not processed for user `017684196c8e49e0894d38daf83bfaeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:25.855090 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-252619ca-b2d4-4c50-b800-cdd6717f7323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:25.864924 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:25.876929 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852e9e5-91a0-4f99-ab38-f6e7f94aa2f8 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:25.881565 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d4f4f-1443-48b6-8b78-6a7bbe1c896e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:25.882384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d4f4f-1443-48b6-8b78-6a7bbe1c896e admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:25.882684 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d4f4f-1443-48b6-8b78-6a7bbe1c896e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:25.882974 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d4f4f-1443-48b6-8b78-6a7bbe1c896e admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:25.883589 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d4f4f-1443-48b6-8b78-6a7bbe1c896e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:25.885489 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d4f4f-1443-48b6-8b78-6a7bbe1c896e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:25.919942 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1138/2295] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:25.934631 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252619ca-b2d4-4c50-b800-cdd6717f7323 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:25.936161 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-252619ca-b2d4-4c50-b800-cdd6717f7323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:25.982896 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1158/2296] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:03:25 2022] POST /identity/v3/projects => generated 383 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:03:25.987132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d4f4f-1443-48b6-8b78-6a7bbe1c896e admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:25.997523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252619ca-b2d4-4c50-b800-cdd6717f7323 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:26.048821 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e69dd7f-616e-4b44-972b-2ce305aaadc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:26.049420 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69dd7f-616e-4b44-972b-2ce305aaadc3 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:26.049697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69dd7f-616e-4b44-972b-2ce305aaadc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:26.049977 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69dd7f-616e-4b44-972b-2ce305aaadc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:26.055773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d4f4f-1443-48b6-8b78-6a7bbe1c896e admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:26.061103 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3aa990-fab7-4830-8011-ea9130286672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:26.061674 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3aa990-fab7-4830-8011-ea9130286672 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:26.061950 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3aa990-fab7-4830-8011-ea9130286672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:26.062236 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3aa990-fab7-4830-8011-ea9130286672 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:26.062859 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3aa990-fab7-4830-8011-ea9130286672 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:26.064987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3aa990-fab7-4830-8011-ea9130286672 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:26.111970 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:26.114720 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e69dd7f-616e-4b44-972b-2ce305aaadc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:26.116155 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e69dd7f-616e-4b44-972b-2ce305aaadc3 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:26.125203 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1139/2297] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:03:26.199291 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1159/2298] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:03:25 2022] POST /identity/v3/users => generated 298 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:03:26.204216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e69dd7f-616e-4b44-972b-2ce305aaadc3 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:26.220832 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e69dd7f-616e-4b44-972b-2ce305aaadc3 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:26.233878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcfbf4-671e-441e-8c31-574cdea1acb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:26.235257 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcfbf4-671e-441e-8c31-574cdea1acb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:26.235545 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcfbf4-671e-441e-8c31-574cdea1acb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:26.235815 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcfbf4-671e-441e-8c31-574cdea1acb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:26.236649 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffcfbf4-671e-441e-8c31-574cdea1acb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:26.238941 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffcfbf4-671e-441e-8c31-574cdea1acb2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:26.267338 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1140/2299] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:03:26.271114 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3aa990-fab7-4830-8011-ea9130286672 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:26.284193 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3aa990-fab7-4830-8011-ea9130286672 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:26.290022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0964e0-68c9-41d7-8a79-d5b9a4e4426d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:26.290210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0964e0-68c9-41d7-8a79-d5b9a4e4426d admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:26.290397 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0964e0-68c9-41d7-8a79-d5b9a4e4426d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:26.290676 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0964e0-68c9-41d7-8a79-d5b9a4e4426d admin admin] PATH_INFO: `/v3/projects/f6ea590d1d8d4e829f99c15b76bcb37a/users/f52eaea6facd4f5294bbde8129ffe8b4/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:26.308038 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0964e0-68c9-41d7-8a79-d5b9a4e4426d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6ea590d1d8d4e829f99c15b76bcb37a, user_id=f52eaea6facd4f5294bbde8129ffe8b4, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:26.310802 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0964e0-68c9-41d7-8a79-d5b9a4e4426d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:26.409422 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1160/2300] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:03:26 2022] PUT /identity/v3/projects/f6ea590d1d8d4e829f99c15b76bcb37a/users/f52eaea6facd4f5294bbde8129ffe8b4/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:26.413418 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcfbf4-671e-441e-8c31-574cdea1acb2 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:26.423167 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcfbf4-671e-441e-8c31-574cdea1acb2 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:26.481035 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90fc5eb-2111-442c-835f-0fc8d8163022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:26.481640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90fc5eb-2111-442c-835f-0fc8d8163022 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:26.481945 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90fc5eb-2111-442c-835f-0fc8d8163022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:26.482188 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90fc5eb-2111-442c-835f-0fc8d8163022 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:26.482870 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90fc5eb-2111-442c-835f-0fc8d8163022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:26.484467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90fc5eb-2111-442c-835f-0fc8d8163022 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:26.495391 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1141/2301] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:03:26.499914 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0964e0-68c9-41d7-8a79-d5b9a4e4426d admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:26.510851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0964e0-68c9-41d7-8a79-d5b9a4e4426d admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:26.515365 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc2400-3084-484b-8897-229aebc6976c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:26.516004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc2400-3084-484b-8897-229aebc6976c admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:26.516282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc2400-3084-484b-8897-229aebc6976c admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:26.516553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc2400-3084-484b-8897-229aebc6976c admin admin] PATH_INFO: `/v3/projects/f6ea590d1d8d4e829f99c15b76bcb37a/users/f52eaea6facd4f5294bbde8129ffe8b4/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:26.518516 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc2400-3084-484b-8897-229aebc6976c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6ea590d1d8d4e829f99c15b76bcb37a, user_id=f52eaea6facd4f5294bbde8129ffe8b4, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:26.521177 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc2400-3084-484b-8897-229aebc6976c admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:26.531566 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1161/2302] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:03:26 2022] PUT /identity/v3/projects/f6ea590d1d8d4e829f99c15b76bcb37a/users/f52eaea6facd4f5294bbde8129ffe8b4/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:26.542730 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9562b2-b208-47cf-8dc5-ffbcc40814f3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:26.543013 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9562b2-b208-47cf-8dc5-ffbcc40814f3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:26.543291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9562b2-b208-47cf-8dc5-ffbcc40814f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:26.571883 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-8d9562b2-b208-47cf-8dc5-ffbcc40814f3 None None] MFA Rules not processed for user `f52eaea6facd4f5294bbde8129ffe8b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:26.573098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8d9562b2-b208-47cf-8dc5-ffbcc40814f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:26.640368 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1142/2303] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:26.651514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc2400-3084-484b-8897-229aebc6976c admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:26.653198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f4fc2400-3084-484b-8897-229aebc6976c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:26.727761 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc2400-3084-484b-8897-229aebc6976c admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:26.732228 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569cc008-0aef-4520-ab1d-cc2a51ad92f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:26.738731 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569cc008-0aef-4520-ab1d-cc2a51ad92f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:26.738731 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569cc008-0aef-4520-ab1d-cc2a51ad92f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:26.738731 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569cc008-0aef-4520-ab1d-cc2a51ad92f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:26.778007 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:26.793351 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569cc008-0aef-4520-ab1d-cc2a51ad92f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:26.795800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569cc008-0aef-4520-ab1d-cc2a51ad92f1 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:26.808292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1162/2304] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:03:28.900151 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9562b2-b208-47cf-8dc5-ffbcc40814f3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:28.909568 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9562b2-b208-47cf-8dc5-ffbcc40814f3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:28.913749 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b1f3a-cc86-4f32-ad32-7609f9212eef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:28.914328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b1f3a-cc86-4f32-ad32-7609f9212eef service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:28.914609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b1f3a-cc86-4f32-ad32-7609f9212eef service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:28.914912 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b1f3a-cc86-4f32-ad32-7609f9212eef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:28.972789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b1f3a-cc86-4f32-ad32-7609f9212eef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:28.974270 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b1f3a-cc86-4f32-ad32-7609f9212eef service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:28.983978 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1143/2305] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:03:29.214685 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d09015-42e4-423e-a842-ca4e728a3552 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:29.215170 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d09015-42e4-423e-a842-ca4e728a3552 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:29.215267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d09015-42e4-423e-a842-ca4e728a3552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:29.235814 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-50d09015-42e4-423e-a842-ca4e728a3552 None None] MFA Rules not processed for user `017684196c8e49e0894d38daf83bfaeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:29.237017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-50d09015-42e4-423e-a842-ca4e728a3552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:29.298636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1163/2306] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:30.490364 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b1f3a-cc86-4f32-ad32-7609f9212eef service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:30.495856 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:30.502216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b1f3a-cc86-4f32-ad32-7609f9212eef service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:30.503813 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5650191-a026-46d2-8aea-1b723692295f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:30.504419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5650191-a026-46d2-8aea-1b723692295f service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:30.504698 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5650191-a026-46d2-8aea-1b723692295f service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:30.504971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5650191-a026-46d2-8aea-1b723692295f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:30.514595 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5650191-a026-46d2-8aea-1b723692295f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:30.516065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5650191-a026-46d2-8aea-1b723692295f service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:30.531790 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1144/2307] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:30 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:03:32.122417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eceeb8a-ad1e-4c12-bf00-80c1e3e87f33 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:32.122939 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eceeb8a-ad1e-4c12-bf00-80c1e3e87f33 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:32.123026 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eceeb8a-ad1e-4c12-bf00-80c1e3e87f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:32.147018 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-9eceeb8a-ad1e-4c12-bf00-80c1e3e87f33 None None] MFA Rules not processed for user `b8ceeb13c4c24ceb9ff1f27b4e45f62c`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:32.149065 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9eceeb8a-ad1e-4c12-bf00-80c1e3e87f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:32.246354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1164/2308] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:32 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:32.257911 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74c1b0-453f-4c5c-b7bb-1b02a93adfa7 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:32.258682 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74c1b0-453f-4c5c-b7bb-1b02a93adfa7 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:32.258682 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74c1b0-453f-4c5c-b7bb-1b02a93adfa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:32.286523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-2a74c1b0-453f-4c5c-b7bb-1b02a93adfa7 None None] MFA Rules not processed for user `b0a879085a194b0687e1cdcbafd48033`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:32.290424 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2a74c1b0-453f-4c5c-b7bb-1b02a93adfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:32.406703 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1145/2309] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Sep 11 00:03:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:03:32.451666 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eceeb8a-ad1e-4c12-bf00-80c1e3e87f33 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:32.453231 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9eceeb8a-ad1e-4c12-bf00-80c1e3e87f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:32.584397 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eceeb8a-ad1e-4c12-bf00-80c1e3e87f33 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:32.592164 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69783b28-9b1f-4179-8d34-6ef0155fb05f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '564e65585e9543e286b6041af870cf41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:32.593080 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69783b28-9b1f-4179-8d34-6ef0155fb05f service swift] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:32.593491 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69783b28-9b1f-4179-8d34-6ef0155fb05f service swift] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:32.593863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69783b28-9b1f-4179-8d34-6ef0155fb05f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:32.605594 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69783b28-9b1f-4179-8d34-6ef0155fb05f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:32.607513 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69783b28-9b1f-4179-8d34-6ef0155fb05f service swift] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:32.620503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1165/2310] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Sep 11 00:03:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 00:03:33.596929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74c1b0-453f-4c5c-b7bb-1b02a93adfa7 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:33.599121 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2a74c1b0-453f-4c5c-b7bb-1b02a93adfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:33.785409 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74c1b0-453f-4c5c-b7bb-1b02a93adfa7 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:33.791921 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82690f9b-9eda-4046-8118-896a636ab1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:33.792877 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82690f9b-9eda-4046-8118-896a636ab1eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:33.793405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82690f9b-9eda-4046-8118-896a636ab1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:33.793874 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82690f9b-9eda-4046-8118-896a636ab1eb admin admin] PATH_INFO: `/v3/users/255694a8bcd041eabb35733448eed42d` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:33.796267 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82690f9b-9eda-4046-8118-896a636ab1eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=255694a8bcd041eabb35733448eed42d)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:33.798962 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82690f9b-9eda-4046-8118-896a636ab1eb admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:34.104395 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-82690f9b-9eda-4046-8118-896a636ab1eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '255694a8bcd041eabb35733448eed42d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:34.119421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-82690f9b-9eda-4046-8118-896a636ab1eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '255694a8bcd041eabb35733448eed42d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:34.149522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-82690f9b-9eda-4046-8118-896a636ab1eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '255694a8bcd041eabb35733448eed42d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:34.234329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-82690f9b-9eda-4046-8118-896a636ab1eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '255694a8bcd041eabb35733448eed42d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:34.235034 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-82690f9b-9eda-4046-8118-896a636ab1eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '255694a8bcd041eabb35733448eed42d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:34.249572 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1146/2311] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:03:33 2022] DELETE /identity/v3/users/255694a8bcd041eabb35733448eed42d => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:34.251344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb5195-156a-4b2b-a0c4-3b4e4e831f5b None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:34.252253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb5195-156a-4b2b-a0c4-3b4e4e831f5b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:34.252712 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb5195-156a-4b2b-a0c4-3b4e4e831f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:34.282797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-6eeb5195-156a-4b2b-a0c4-3b4e4e831f5b None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:34.284726 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6eeb5195-156a-4b2b-a0c4-3b4e4e831f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:34.324110 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:34.440405 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1166/2312] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:03:34.459049 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82690f9b-9eda-4046-8118-896a636ab1eb admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:34.461822 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-82690f9b-9eda-4046-8118-896a636ab1eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:34.496271 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:34.547525 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82690f9b-9eda-4046-8118-896a636ab1eb admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:34.552486 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958d669-cc7f-4bcd-ae76-79ff491298f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:34.553162 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958d669-cc7f-4bcd-ae76-79ff491298f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:34.554488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958d669-cc7f-4bcd-ae76-79ff491298f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:34.554711 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958d669-cc7f-4bcd-ae76-79ff491298f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:34.567591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958d669-cc7f-4bcd-ae76-79ff491298f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:34.571206 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958d669-cc7f-4bcd-ae76-79ff491298f0 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:34.588915 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1147/2313] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:03:35.780826 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeb5195-156a-4b2b-a0c4-3b4e4e831f5b None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:35.782415 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6eeb5195-156a-4b2b-a0c4-3b4e4e831f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:35.798203 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeb5195-156a-4b2b-a0c4-3b4e4e831f5b None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:35.803058 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80afb7f-67c0-40af-8a72-134d10740222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:35.804704 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80afb7f-67c0-40af-8a72-134d10740222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:35.805335 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80afb7f-67c0-40af-8a72-134d10740222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:35.805806 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80afb7f-67c0-40af-8a72-134d10740222 admin admin] PATH_INFO: `/v3/projects/96a4c885eada487d849017e4e6e981eb` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:35.807207 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80afb7f-67c0-40af-8a72-134d10740222 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96a4c885eada487d849017e4e6e981eb)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:35.814953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80afb7f-67c0-40af-8a72-134d10740222 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:35.943508 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c80afb7f-67c0-40af-8a72-134d10740222 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96a4c885eada487d849017e4e6e981eb', 'request_id': 'req-c80afb7f-67c0-40af-8a72-134d10740222'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:36.021384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c80afb7f-67c0-40af-8a72-134d10740222 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96a4c885eada487d849017e4e6e981eb', 'request_id': 'req-c80afb7f-67c0-40af-8a72-134d10740222'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:36.030967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c80afb7f-67c0-40af-8a72-134d10740222 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96a4c885eada487d849017e4e6e981eb', 'request_id': 'req-c80afb7f-67c0-40af-8a72-134d10740222'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:36.049875 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c80afb7f-67c0-40af-8a72-134d10740222 admin admin] The token cache is being invalidate because project 96a4c885eada487d849017e4e6e981eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:03:36.050087 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c80afb7f-67c0-40af-8a72-134d10740222 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:36.051251 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c80afb7f-67c0-40af-8a72-134d10740222 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:36.053099 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1167/2314] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:03:35 2022] DELETE /identity/v3/projects/96a4c885eada487d849017e4e6e981eb => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:36.058170 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07599e2-ad8e-4ddc-8b33-21c9b062fa64 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:36.059085 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07599e2-ad8e-4ddc-8b33-21c9b062fa64 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:36.059353 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07599e2-ad8e-4ddc-8b33-21c9b062fa64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:36.086241 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-e07599e2-ad8e-4ddc-8b33-21c9b062fa64 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:36.087943 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e07599e2-ad8e-4ddc-8b33-21c9b062fa64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:36.197513 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1148/2315] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:36.205074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce70805-c4b1-41bb-b75f-b34f1982abff None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:36.205702 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce70805-c4b1-41bb-b75f-b34f1982abff None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:36.206205 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce70805-c4b1-41bb-b75f-b34f1982abff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:36.242162 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-3ce70805-c4b1-41bb-b75f-b34f1982abff None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:36.243467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3ce70805-c4b1-41bb-b75f-b34f1982abff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:36.266282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1168/2316] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:36.270922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07599e2-ad8e-4ddc-8b33-21c9b062fa64 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:36.300143 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07599e2-ad8e-4ddc-8b33-21c9b062fa64 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:36.317484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd30570-942c-4aa2-a72a-d169b0d47969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:36.318484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd30570-942c-4aa2-a72a-d169b0d47969 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:36.318846 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd30570-942c-4aa2-a72a-d169b0d47969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:36.319814 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd30570-942c-4aa2-a72a-d169b0d47969 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:36.320573 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd30570-942c-4aa2-a72a-d169b0d47969 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:03:36.321157 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd30570-942c-4aa2-a72a-d169b0d47969 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:36.322986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd30570-942c-4aa2-a72a-d169b0d47969 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:36.344830 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce70805-c4b1-41bb-b75f-b34f1982abff None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:36.351454 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1149/2317] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:03:36.355628 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce70805-c4b1-41bb-b75f-b34f1982abff None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:36.361592 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49736c8-e090-48bd-a593-27e54f709a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:36.362615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49736c8-e090-48bd-a593-27e54f709a12 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:36.363165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49736c8-e090-48bd-a593-27e54f709a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:36.363647 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49736c8-e090-48bd-a593-27e54f709a12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:36.364491 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49736c8-e090-48bd-a593-27e54f709a12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:36.367957 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49736c8-e090-48bd-a593-27e54f709a12 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:36.469185 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1169/2318] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:03:36 2022] POST /identity/v3/projects => generated 371 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:03:36.470833 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd30570-942c-4aa2-a72a-d169b0d47969 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:36.482512 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd30570-942c-4aa2-a72a-d169b0d47969 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:36.662374 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8bd5b-f3bc-4a6d-bd2b-6b5de62dbdfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:36.662374 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8bd5b-f3bc-4a6d-bd2b-6b5de62dbdfe admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:36.662374 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8bd5b-f3bc-4a6d-bd2b-6b5de62dbdfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:36.662374 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8bd5b-f3bc-4a6d-bd2b-6b5de62dbdfe admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:36.663908 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8bd5b-f3bc-4a6d-bd2b-6b5de62dbdfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:36.663908 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8bd5b-f3bc-4a6d-bd2b-6b5de62dbdfe admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:36.895334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1150/2319] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:03:36 2022] POST /identity/v3/users => generated 286 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:03:36.902430 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49736c8-e090-48bd-a593-27e54f709a12 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:36.913166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49736c8-e090-48bd-a593-27e54f709a12 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:36.922436 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3356c-5219-4af8-8ed5-53425c02d8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:36.930059 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3356c-5219-4af8-8ed5-53425c02d8c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:36.930059 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3356c-5219-4af8-8ed5-53425c02d8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:36.930059 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3356c-5219-4af8-8ed5-53425c02d8c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:36.930059 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d3356c-5219-4af8-8ed5-53425c02d8c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:36.938400 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d3356c-5219-4af8-8ed5-53425c02d8c2 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:36.962748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1170/2320] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:03:36.972238 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8bd5b-f3bc-4a6d-bd2b-6b5de62dbdfe admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:36.985253 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8bd5b-f3bc-4a6d-bd2b-6b5de62dbdfe admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:36.990136 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4003408-0027-4822-bf4f-ba3d5cc53029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:36.991450 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4003408-0027-4822-bf4f-ba3d5cc53029 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:36.991844 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4003408-0027-4822-bf4f-ba3d5cc53029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:36.992146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4003408-0027-4822-bf4f-ba3d5cc53029 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:36.992769 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4003408-0027-4822-bf4f-ba3d5cc53029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:36.994392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4003408-0027-4822-bf4f-ba3d5cc53029 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:37.010764 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1151/2321] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:03:37.012021 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3356c-5219-4af8-8ed5-53425c02d8c2 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:37.029994 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3356c-5219-4af8-8ed5-53425c02d8c2 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:37.037157 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c437a95-d153-4140-9627-2caf0746234f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:37.039515 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c437a95-d153-4140-9627-2caf0746234f admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:37.039998 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c437a95-d153-4140-9627-2caf0746234f admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:37.040384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c437a95-d153-4140-9627-2caf0746234f admin admin] PATH_INFO: `/v3/projects/eeafb76d202a4092927dac944edb37ae/users/d59b6eaa9f3f48809050fb1fab284301/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:37.057176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c437a95-d153-4140-9627-2caf0746234f admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeafb76d202a4092927dac944edb37ae, user_id=d59b6eaa9f3f48809050fb1fab284301, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:37.059886 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c437a95-d153-4140-9627-2caf0746234f admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:37.120842 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1171/2322] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:03:37 2022] PUT /identity/v3/projects/eeafb76d202a4092927dac944edb37ae/users/d59b6eaa9f3f48809050fb1fab284301/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:37.132823 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefccc17-ab65-4de2-88b2-9c936c29c886 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:37.133396 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefccc17-ab65-4de2-88b2-9c936c29c886 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:37.133396 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefccc17-ab65-4de2-88b2-9c936c29c886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:37.166014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-eefccc17-ab65-4de2-88b2-9c936c29c886 None None] MFA Rules not processed for user `d59b6eaa9f3f48809050fb1fab284301`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:37.167189 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-eefccc17-ab65-4de2-88b2-9c936c29c886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:37.188049 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:37.266494 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1152/2323] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:37.275026 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c437a95-d153-4140-9627-2caf0746234f admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:37.276874 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2c437a95-d153-4140-9627-2caf0746234f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:37.376587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:37.396182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c437a95-d153-4140-9627-2caf0746234f admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:37.401041 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e68d64-c17f-466c-a11a-b358c3654dfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:37.401944 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e68d64-c17f-466c-a11a-b358c3654dfa service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:37.402420 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e68d64-c17f-466c-a11a-b358c3654dfa service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:37.402946 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e68d64-c17f-466c-a11a-b358c3654dfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:37.415417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e68d64-c17f-466c-a11a-b358c3654dfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:37.417276 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e68d64-c17f-466c-a11a-b358c3654dfa service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:37.432383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1172/2324] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:03:37.474966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefccc17-ab65-4de2-88b2-9c936c29c886 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:37.626957 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefccc17-ab65-4de2-88b2-9c936c29c886 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:37.633231 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db745f9d-c8a7-4f53-a084-49287f2b6c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:37.633877 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db745f9d-c8a7-4f53-a084-49287f2b6c79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:37.634153 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db745f9d-c8a7-4f53-a084-49287f2b6c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:37.634422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db745f9d-c8a7-4f53-a084-49287f2b6c79 admin admin] PATH_INFO: `/v3/users/d59b6eaa9f3f48809050fb1fab284301` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:37.635671 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db745f9d-c8a7-4f53-a084-49287f2b6c79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d59b6eaa9f3f48809050fb1fab284301)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:37.637916 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db745f9d-c8a7-4f53-a084-49287f2b6c79 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:37.911857 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-db745f9d-c8a7-4f53-a084-49287f2b6c79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd59b6eaa9f3f48809050fb1fab284301'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:37.936932 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-db745f9d-c8a7-4f53-a084-49287f2b6c79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd59b6eaa9f3f48809050fb1fab284301'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:37.962745 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-db745f9d-c8a7-4f53-a084-49287f2b6c79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd59b6eaa9f3f48809050fb1fab284301'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:38.024768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-db745f9d-c8a7-4f53-a084-49287f2b6c79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd59b6eaa9f3f48809050fb1fab284301'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:38.026134 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-db745f9d-c8a7-4f53-a084-49287f2b6c79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd59b6eaa9f3f48809050fb1fab284301'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:38.039853 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1153/2325] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:03:37 2022] DELETE /identity/v3/users/d59b6eaa9f3f48809050fb1fab284301 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:38.041536 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975a5a0-c3f0-49d3-b772-9aa147a015d4 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:38.042005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975a5a0-c3f0-49d3-b772-9aa147a015d4 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:38.042108 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975a5a0-c3f0-49d3-b772-9aa147a015d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:38.075492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-9975a5a0-c3f0-49d3-b772-9aa147a015d4 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:38.076945 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9975a5a0-c3f0-49d3-b772-9aa147a015d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:38.178954 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1173/2326] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:38.190057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db745f9d-c8a7-4f53-a084-49287f2b6c79 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:38.192640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-db745f9d-c8a7-4f53-a084-49287f2b6c79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:38.270563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:38.311109 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db745f9d-c8a7-4f53-a084-49287f2b6c79 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:38.317417 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d996055c-6198-4ad3-9d17-a15a740d7a4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:38.318033 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996055c-6198-4ad3-9d17-a15a740d7a4d service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:38.321305 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996055c-6198-4ad3-9d17-a15a740d7a4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:38.321305 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996055c-6198-4ad3-9d17-a15a740d7a4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:38.341859 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d996055c-6198-4ad3-9d17-a15a740d7a4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:38.344718 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d996055c-6198-4ad3-9d17-a15a740d7a4d service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:38.357264 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1154/2327] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:03:39.086239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9975a5a0-c3f0-49d3-b772-9aa147a015d4 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:39.086239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9975a5a0-c3f0-49d3-b772-9aa147a015d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:39.106698 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9975a5a0-c3f0-49d3-b772-9aa147a015d4 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:39.114778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52637884-af49-4ba1-9b4d-48d975f4f44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:39.118114 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52637884-af49-4ba1-9b4d-48d975f4f44a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:39.118459 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52637884-af49-4ba1-9b4d-48d975f4f44a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:39.118791 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52637884-af49-4ba1-9b4d-48d975f4f44a admin admin] PATH_INFO: `/v3/projects/eeafb76d202a4092927dac944edb37ae` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:39.120362 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52637884-af49-4ba1-9b4d-48d975f4f44a admin admin] RBAC: Authorizing `identity:delete_project(project_id=eeafb76d202a4092927dac944edb37ae)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:39.125065 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52637884-af49-4ba1-9b4d-48d975f4f44a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:39.210913 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d996055c-6198-4ad3-9d17-a15a740d7a4d service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:39.212506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d996055c-6198-4ad3-9d17-a15a740d7a4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:39.239453 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d996055c-6198-4ad3-9d17-a15a740d7a4d service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:39.244958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee413c-fcf5-44e9-a07f-57a987ac31a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:39.245916 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee413c-fcf5-44e9-a07f-57a987ac31a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:39.246414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee413c-fcf5-44e9-a07f-57a987ac31a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:39.246926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee413c-fcf5-44e9-a07f-57a987ac31a0 admin admin] PATH_INFO: `/v3/users/27036d13b9fb49798b8e5606bc422181` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:39.248615 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee413c-fcf5-44e9-a07f-57a987ac31a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27036d13b9fb49798b8e5606bc422181)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:39.251143 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee413c-fcf5-44e9-a07f-57a987ac31a0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:39.308158 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-52637884-af49-4ba1-9b4d-48d975f4f44a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eeafb76d202a4092927dac944edb37ae', 'request_id': 'req-52637884-af49-4ba1-9b4d-48d975f4f44a'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:39.423326 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-52637884-af49-4ba1-9b4d-48d975f4f44a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eeafb76d202a4092927dac944edb37ae', 'request_id': 'req-52637884-af49-4ba1-9b4d-48d975f4f44a'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:39.434085 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-52637884-af49-4ba1-9b4d-48d975f4f44a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eeafb76d202a4092927dac944edb37ae', 'request_id': 'req-52637884-af49-4ba1-9b4d-48d975f4f44a'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:39.454044 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-52637884-af49-4ba1-9b4d-48d975f4f44a admin admin] The token cache is being invalidate because project eeafb76d202a4092927dac944edb37ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:03:39.456068 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-52637884-af49-4ba1-9b4d-48d975f4f44a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:39.456068 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-52637884-af49-4ba1-9b4d-48d975f4f44a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:39.462508 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1174/2328] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:03:39 2022] DELETE /identity/v3/projects/eeafb76d202a4092927dac944edb37ae => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:39.464579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cda04c-9cc3-4316-bc68-54b250fa162a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:39.465149 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cda04c-9cc3-4316-bc68-54b250fa162a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:39.465665 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cda04c-9cc3-4316-bc68-54b250fa162a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:39.496202 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1aee413c-fcf5-44e9-a07f-57a987ac31a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27036d13b9fb49798b8e5606bc422181'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:39.500967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-16cda04c-9cc3-4316-bc68-54b250fa162a None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:39.504087 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-16cda04c-9cc3-4316-bc68-54b250fa162a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:39.508315 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1aee413c-fcf5-44e9-a07f-57a987ac31a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27036d13b9fb49798b8e5606bc422181'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:39.529737 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1aee413c-fcf5-44e9-a07f-57a987ac31a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27036d13b9fb49798b8e5606bc422181'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:39.616773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1175/2329] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:39.618847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be3714-526d-4093-831b-b59e056f9b1a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:39.619138 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be3714-526d-4093-831b-b59e056f9b1a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:39.619417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be3714-526d-4093-831b-b59e056f9b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:39.624801 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1aee413c-fcf5-44e9-a07f-57a987ac31a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27036d13b9fb49798b8e5606bc422181'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:39.625753 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-1aee413c-fcf5-44e9-a07f-57a987ac31a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27036d13b9fb49798b8e5606bc422181'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:39.637904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1155/2330] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:03:39 2022] DELETE /identity/v3/users/27036d13b9fb49798b8e5606bc422181 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:39.645291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cdca7b-5033-4633-95a2-3c803d4e45b3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:39.645850 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cdca7b-5033-4633-95a2-3c803d4e45b3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:39.646296 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cdca7b-5033-4633-95a2-3c803d4e45b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:39.655927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-f6be3714-526d-4093-831b-b59e056f9b1a None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:39.659271 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f6be3714-526d-4093-831b-b59e056f9b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:39.665801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:39.685734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-09cdca7b-5033-4633-95a2-3c803d4e45b3 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:39.687970 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-09cdca7b-5033-4633-95a2-3c803d4e45b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:39.795738 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1176/2331] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:39.804804 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be3714-526d-4093-831b-b59e056f9b1a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:39.831508 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be3714-526d-4093-831b-b59e056f9b1a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:39.844455 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af8c26-e7dd-4039-9b2e-fd19bf42fade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:39.845662 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af8c26-e7dd-4039-9b2e-fd19bf42fade admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:39.845858 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af8c26-e7dd-4039-9b2e-fd19bf42fade admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:39.846478 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af8c26-e7dd-4039-9b2e-fd19bf42fade admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:39.847753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af8c26-e7dd-4039-9b2e-fd19bf42fade admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:03:39.848173 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af8c26-e7dd-4039-9b2e-fd19bf42fade admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:39.849960 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af8c26-e7dd-4039-9b2e-fd19bf42fade admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:39.867724 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1156/2332] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:39.883153 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cdca7b-5033-4633-95a2-3c803d4e45b3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:39.884654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1177/2333] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:03:39.886604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af8c26-e7dd-4039-9b2e-fd19bf42fade admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:39.888463 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a8af8c26-e7dd-4039-9b2e-fd19bf42fade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:39.902474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cdca7b-5033-4633-95a2-3c803d4e45b3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:39.918500 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f9de6b-21c6-4482-b96a-4388dc383b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:39.920001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9de6b-21c6-4482-b96a-4388dc383b3f admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:39.924973 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9de6b-21c6-4482-b96a-4388dc383b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:39.924973 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9de6b-21c6-4482-b96a-4388dc383b3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:39.924973 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f9de6b-21c6-4482-b96a-4388dc383b3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:39.925413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f9de6b-21c6-4482-b96a-4388dc383b3f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:40.001774 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af8c26-e7dd-4039-9b2e-fd19bf42fade admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:40.007980 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1720695-3eea-4fa7-b418-b2186fef5a3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:40.009485 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1720695-3eea-4fa7-b418-b2186fef5a3e service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:40.013201 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1720695-3eea-4fa7-b418-b2186fef5a3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:40.013201 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1720695-3eea-4fa7-b418-b2186fef5a3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:40.023432 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1720695-3eea-4fa7-b418-b2186fef5a3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:40.025638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1720695-3eea-4fa7-b418-b2186fef5a3e service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:40.078922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1157/2334] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:03:39 2022] POST /identity/v3/projects => generated 375 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:03:40.079806 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f9de6b-21c6-4482-b96a-4388dc383b3f admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:40.088429 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1178/2335] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:03:40.164714 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:40.282120 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f9de6b-21c6-4482-b96a-4388dc383b3f admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:40.293182 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf32411-e228-48cb-8fe1-a852fd5e9770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:40.295919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf32411-e228-48cb-8fe1-a852fd5e9770 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:40.296820 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf32411-e228-48cb-8fe1-a852fd5e9770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:40.297477 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf32411-e228-48cb-8fe1-a852fd5e9770 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:40.298406 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf32411-e228-48cb-8fe1-a852fd5e9770 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:40.308226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf32411-e228-48cb-8fe1-a852fd5e9770 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:40.455082 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1158/2336] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:03:40 2022] POST /identity/v3/users => generated 288 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:03:40.459977 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1720695-3eea-4fa7-b418-b2186fef5a3e service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:40.478564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1720695-3eea-4fa7-b418-b2186fef5a3e service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:40.485291 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4fa27-4989-4e68-a704-f47398c1bd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:40.486127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4fa27-4989-4e68-a704-f47398c1bd15 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:40.486479 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4fa27-4989-4e68-a704-f47398c1bd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:40.486861 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4fa27-4989-4e68-a704-f47398c1bd15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:40.521683 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a4fa27-4989-4e68-a704-f47398c1bd15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:40.526325 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a4fa27-4989-4e68-a704-f47398c1bd15 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:40.551083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1179/2337] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:03:40.554785 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf32411-e228-48cb-8fe1-a852fd5e9770 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:40.578711 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf32411-e228-48cb-8fe1-a852fd5e9770 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:40.591357 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91932ba-8c63-48d8-8778-2c743a2e7cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:40.599218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91932ba-8c63-48d8-8778-2c743a2e7cac admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:40.599218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91932ba-8c63-48d8-8778-2c743a2e7cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:40.599218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91932ba-8c63-48d8-8778-2c743a2e7cac admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:40.599218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91932ba-8c63-48d8-8778-2c743a2e7cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:40.599218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91932ba-8c63-48d8-8778-2c743a2e7cac admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:40.621876 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1159/2338] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:03:40.624563 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4fa27-4989-4e68-a704-f47398c1bd15 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:40.645833 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4fa27-4989-4e68-a704-f47398c1bd15 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:40.660216 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e008ec7-d939-4b7a-9348-02aecc130201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:40.662575 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e008ec7-d939-4b7a-9348-02aecc130201 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:40.663275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e008ec7-d939-4b7a-9348-02aecc130201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:40.664147 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e008ec7-d939-4b7a-9348-02aecc130201 admin admin] PATH_INFO: `/v3/projects/8f2e66e3f8ee4e85960d55afde84ef7a/users/ffd55dbc683b4bcf8e9a3ae82e07cf3d/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:40.721782 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e008ec7-d939-4b7a-9348-02aecc130201 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f2e66e3f8ee4e85960d55afde84ef7a, user_id=ffd55dbc683b4bcf8e9a3ae82e07cf3d, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:40.728717 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e008ec7-d939-4b7a-9348-02aecc130201 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:40.833022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1180/2339] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:03:40 2022] PUT /identity/v3/projects/8f2e66e3f8ee4e85960d55afde84ef7a/users/ffd55dbc683b4bcf8e9a3ae82e07cf3d/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:40.833447 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3651d4-fe3b-4d76-bc6c-c22235a574e0 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:40.833732 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3651d4-fe3b-4d76-bc6c-c22235a574e0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:40.834008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3651d4-fe3b-4d76-bc6c-c22235a574e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:40.863512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e008ec7-d939-4b7a-9348-02aecc130201 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:40.869359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-7e008ec7-d939-4b7a-9348-02aecc130201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:40.900093 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-6f3651d4-fe3b-4d76-bc6c-c22235a574e0 None None] MFA Rules not processed for user `ffd55dbc683b4bcf8e9a3ae82e07cf3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:40.910506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6f3651d4-fe3b-4d76-bc6c-c22235a574e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:41.064771 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e008ec7-d939-4b7a-9348-02aecc130201 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:41.072159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a5c7c-25f3-4ea2-90f7-e97e69ace183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:41.072814 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a5c7c-25f3-4ea2-90f7-e97e69ace183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:41.073088 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a5c7c-25f3-4ea2-90f7-e97e69ace183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:41.073357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a5c7c-25f3-4ea2-90f7-e97e69ace183 admin admin] PATH_INFO: `/v3/projects/026452f1f397459fb8f7cf463a9146f6` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:41.074711 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a5c7c-25f3-4ea2-90f7-e97e69ace183 admin admin] RBAC: Authorizing `identity:delete_project(project_id=026452f1f397459fb8f7cf463a9146f6)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:41.076636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a5c7c-25f3-4ea2-90f7-e97e69ace183 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:41.090164 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:41.136748 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1160/2340] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:03:41.156330 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3651d4-fe3b-4d76-bc6c-c22235a574e0 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:41.159128 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6f3651d4-fe3b-4d76-bc6c-c22235a574e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:41.317149 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3651d4-fe3b-4d76-bc6c-c22235a574e0 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:41.319158 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ce5a5c7c-25f3-4ea2-90f7-e97e69ace183 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '026452f1f397459fb8f7cf463a9146f6', 'request_id': 'req-ce5a5c7c-25f3-4ea2-90f7-e97e69ace183'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:41.418434 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ce5a5c7c-25f3-4ea2-90f7-e97e69ace183 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '026452f1f397459fb8f7cf463a9146f6', 'request_id': 'req-ce5a5c7c-25f3-4ea2-90f7-e97e69ace183'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:41.426499 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ce5a5c7c-25f3-4ea2-90f7-e97e69ace183 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '026452f1f397459fb8f7cf463a9146f6', 'request_id': 'req-ce5a5c7c-25f3-4ea2-90f7-e97e69ace183'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:41.452005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ce5a5c7c-25f3-4ea2-90f7-e97e69ace183 admin admin] The token cache is being invalidate because project 026452f1f397459fb8f7cf463a9146f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:03:41.452223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ce5a5c7c-25f3-4ea2-90f7-e97e69ace183 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:41.452921 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ce5a5c7c-25f3-4ea2-90f7-e97e69ace183 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:41.453422 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550dc61c-ca96-4ac4-bd4a-724f8d11517d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:41.454862 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1181/2341] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:03:40 2022] DELETE /identity/v3/projects/026452f1f397459fb8f7cf463a9146f6 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:41.459048 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a5c7c-25f3-4ea2-90f7-e97e69ace183 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:41.461222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dc61c-ca96-4ac4-bd4a-724f8d11517d service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:41.464828 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ce5a5c7c-25f3-4ea2-90f7-e97e69ace183 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:41.464962 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dc61c-ca96-4ac4-bd4a-724f8d11517d service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:41.464962 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dc61c-ca96-4ac4-bd4a-724f8d11517d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:41.472408 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-550dc61c-ca96-4ac4-bd4a-724f8d11517d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:41.485542 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:41.573098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a5c7c-25f3-4ea2-90f7-e97e69ace183 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:41.578848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b51d7-44c6-40c1-be20-9b91cb871f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:41.579627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b51d7-44c6-40c1-be20-9b91cb871f7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:41.579902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b51d7-44c6-40c1-be20-9b91cb871f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:41.580285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b51d7-44c6-40c1-be20-9b91cb871f7b admin admin] PATH_INFO: `/v3/users/fa796f6f4b6643738fddaf2d0c1651d8` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:41.581340 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367b51d7-44c6-40c1-be20-9b91cb871f7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa796f6f4b6643738fddaf2d0c1651d8)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:41.583325 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367b51d7-44c6-40c1-be20-9b91cb871f7b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:41.605446 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550dc61c-ca96-4ac4-bd4a-724f8d11517d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:41.606850 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550dc61c-ca96-4ac4-bd4a-724f8d11517d service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:41.640708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1161/2342] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:03:42.423490 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-367b51d7-44c6-40c1-be20-9b91cb871f7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa796f6f4b6643738fddaf2d0c1651d8'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:42.437166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-367b51d7-44c6-40c1-be20-9b91cb871f7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa796f6f4b6643738fddaf2d0c1651d8'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:42.444932 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-367b51d7-44c6-40c1-be20-9b91cb871f7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa796f6f4b6643738fddaf2d0c1651d8'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:42.445658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-367b51d7-44c6-40c1-be20-9b91cb871f7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa796f6f4b6643738fddaf2d0c1651d8'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:42.529907 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-367b51d7-44c6-40c1-be20-9b91cb871f7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa796f6f4b6643738fddaf2d0c1651d8'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:42.537289 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1182/2343] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:03:41 2022] DELETE /identity/v3/users/fa796f6f4b6643738fddaf2d0c1651d8 => generated 0 bytes in 1079 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:03:43.455139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550dc61c-ca96-4ac4-bd4a-724f8d11517d service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:43.505471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550dc61c-ca96-4ac4-bd4a-724f8d11517d service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:43.643900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aeb4b8-a708-458b-b1e6-ce6d80cf9c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:43.644552 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aeb4b8-a708-458b-b1e6-ce6d80cf9c53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:43.644829 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aeb4b8-a708-458b-b1e6-ce6d80cf9c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:43.645100 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aeb4b8-a708-458b-b1e6-ce6d80cf9c53 admin admin] PATH_INFO: `/v3/projects/32d405ad662a43e1924b6bd3448f014d` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:43.646420 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aeb4b8-a708-458b-b1e6-ce6d80cf9c53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32d405ad662a43e1924b6bd3448f014d)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:43.649523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aeb4b8-a708-458b-b1e6-ce6d80cf9c53 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:43.831313 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f3aeb4b8-a708-458b-b1e6-ce6d80cf9c53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32d405ad662a43e1924b6bd3448f014d', 'request_id': 'req-f3aeb4b8-a708-458b-b1e6-ce6d80cf9c53'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:43.844759 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f3aeb4b8-a708-458b-b1e6-ce6d80cf9c53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32d405ad662a43e1924b6bd3448f014d', 'request_id': 'req-f3aeb4b8-a708-458b-b1e6-ce6d80cf9c53'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:43.852849 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f3aeb4b8-a708-458b-b1e6-ce6d80cf9c53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32d405ad662a43e1924b6bd3448f014d', 'request_id': 'req-f3aeb4b8-a708-458b-b1e6-ce6d80cf9c53'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:43.983570 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f3aeb4b8-a708-458b-b1e6-ce6d80cf9c53 admin admin] The token cache is being invalidate because project 32d405ad662a43e1924b6bd3448f014d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:03:43.983570 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f3aeb4b8-a708-458b-b1e6-ce6d80cf9c53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:43.983570 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f3aeb4b8-a708-458b-b1e6-ce6d80cf9c53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:43.983570 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1162/2344] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:03:43 2022] DELETE /identity/v3/projects/32d405ad662a43e1924b6bd3448f014d => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:43.985798 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b51d7-44c6-40c1-be20-9b91cb871f7b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:43.987345 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-367b51d7-44c6-40c1-be20-9b91cb871f7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:44.002315 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b51d7-44c6-40c1-be20-9b91cb871f7b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:44.024077 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:44.176784 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb2af9-c343-4dbc-801a-0c5c25686194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:44.177517 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb2af9-c343-4dbc-801a-0c5c25686194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:44.177835 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb2af9-c343-4dbc-801a-0c5c25686194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:44.183540 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb2af9-c343-4dbc-801a-0c5c25686194 admin admin] PATH_INFO: `/v3/users/f52eaea6facd4f5294bbde8129ffe8b4` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:44.183540 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb2af9-c343-4dbc-801a-0c5c25686194 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f52eaea6facd4f5294bbde8129ffe8b4)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:44.187703 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb2af9-c343-4dbc-801a-0c5c25686194 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:44.454330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-fbcb2af9-c343-4dbc-801a-0c5c25686194 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f52eaea6facd4f5294bbde8129ffe8b4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:44.489357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-fbcb2af9-c343-4dbc-801a-0c5c25686194 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f52eaea6facd4f5294bbde8129ffe8b4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:44.525184 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-fbcb2af9-c343-4dbc-801a-0c5c25686194 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f52eaea6facd4f5294bbde8129ffe8b4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:44.526508 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-fbcb2af9-c343-4dbc-801a-0c5c25686194 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f52eaea6facd4f5294bbde8129ffe8b4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:44.586475 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-fbcb2af9-c343-4dbc-801a-0c5c25686194 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f52eaea6facd4f5294bbde8129ffe8b4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:44.597445 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1183/2345] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:03:43 2022] DELETE /identity/v3/users/f52eaea6facd4f5294bbde8129ffe8b4 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:45.703886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aeb4b8-a708-458b-b1e6-ce6d80cf9c53 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:45.703886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f3aeb4b8-a708-458b-b1e6-ce6d80cf9c53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:45.723547 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aeb4b8-a708-458b-b1e6-ce6d80cf9c53 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:45.781876 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:45.814336 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a27cc-76b6-416c-b9c3-495c408068d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:45.814818 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a27cc-76b6-416c-b9c3-495c408068d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:45.815410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a27cc-76b6-416c-b9c3-495c408068d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:45.816755 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a27cc-76b6-416c-b9c3-495c408068d4 admin admin] PATH_INFO: `/v3/projects/f6ea590d1d8d4e829f99c15b76bcb37a` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:45.822023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01a27cc-76b6-416c-b9c3-495c408068d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6ea590d1d8d4e829f99c15b76bcb37a)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:45.825894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01a27cc-76b6-416c-b9c3-495c408068d4 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:45.958494 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a01a27cc-76b6-416c-b9c3-495c408068d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6ea590d1d8d4e829f99c15b76bcb37a', 'request_id': 'req-a01a27cc-76b6-416c-b9c3-495c408068d4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:45.965927 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a01a27cc-76b6-416c-b9c3-495c408068d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6ea590d1d8d4e829f99c15b76bcb37a', 'request_id': 'req-a01a27cc-76b6-416c-b9c3-495c408068d4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:45.971985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a01a27cc-76b6-416c-b9c3-495c408068d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6ea590d1d8d4e829f99c15b76bcb37a', 'request_id': 'req-a01a27cc-76b6-416c-b9c3-495c408068d4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:46.064638 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a01a27cc-76b6-416c-b9c3-495c408068d4 admin admin] The token cache is being invalidate because project f6ea590d1d8d4e829f99c15b76bcb37a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:03:46.064930 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a01a27cc-76b6-416c-b9c3-495c408068d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:46.065578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a01a27cc-76b6-416c-b9c3-495c408068d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:46.067329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1163/2346] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:03:45 2022] DELETE /identity/v3/projects/f6ea590d1d8d4e829f99c15b76bcb37a => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:54.705335 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb2af9-c343-4dbc-801a-0c5c25686194 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:54.708098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-fbcb2af9-c343-4dbc-801a-0c5c25686194 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:54.721584 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb2af9-c343-4dbc-801a-0c5c25686194 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:54.791313 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295133b1-434b-4df5-a026-6f1095b06a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:54.791956 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295133b1-434b-4df5-a026-6f1095b06a00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:54.792237 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295133b1-434b-4df5-a026-6f1095b06a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:54.792515 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295133b1-434b-4df5-a026-6f1095b06a00 admin admin] PATH_INFO: `/v3/users/dc663fa6f7274749a9b8263ee7229e2f` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:54.793727 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295133b1-434b-4df5-a026-6f1095b06a00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc663fa6f7274749a9b8263ee7229e2f)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:54.796344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295133b1-434b-4df5-a026-6f1095b06a00 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:55.022219 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:55.031792 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-295133b1-434b-4df5-a026-6f1095b06a00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc663fa6f7274749a9b8263ee7229e2f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:55.044374 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-295133b1-434b-4df5-a026-6f1095b06a00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc663fa6f7274749a9b8263ee7229e2f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:55.051179 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-295133b1-434b-4df5-a026-6f1095b06a00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc663fa6f7274749a9b8263ee7229e2f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:55.051876 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-295133b1-434b-4df5-a026-6f1095b06a00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc663fa6f7274749a9b8263ee7229e2f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:55.149931 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-295133b1-434b-4df5-a026-6f1095b06a00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc663fa6f7274749a9b8263ee7229e2f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:55.157073 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1184/2347] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:03:54 2022] DELETE /identity/v3/users/dc663fa6f7274749a9b8263ee7229e2f => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:55.161456 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb05ee2-b58a-4c38-bfce-68682b573e24 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:55.162028 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb05ee2-b58a-4c38-bfce-68682b573e24 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:55.162028 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb05ee2-b58a-4c38-bfce-68682b573e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:55.182850 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-8eb05ee2-b58a-4c38-bfce-68682b573e24 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:55.185342 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8eb05ee2-b58a-4c38-bfce-68682b573e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:55.253300 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1164/2348] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:55.260484 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295133b1-434b-4df5-a026-6f1095b06a00 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:55.265786 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-295133b1-434b-4df5-a026-6f1095b06a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:55.331309 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295133b1-434b-4df5-a026-6f1095b06a00 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:55.337188 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59a535-86af-48a7-850a-827a993732a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:55.337979 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59a535-86af-48a7-850a-827a993732a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:55.338138 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59a535-86af-48a7-850a-827a993732a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:55.338409 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59a535-86af-48a7-850a-827a993732a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:55.349091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59a535-86af-48a7-850a-827a993732a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:55.350429 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59a535-86af-48a7-850a-827a993732a4 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:55.360326 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1185/2349] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:03:55.876425 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb05ee2-b58a-4c38-bfce-68682b573e24 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:55.878026 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8eb05ee2-b58a-4c38-bfce-68682b573e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:55.893407 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb05ee2-b58a-4c38-bfce-68682b573e24 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:55.899338 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303d6b28-f2d8-4ebd-8ee2-60920533184b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:55.900006 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d6b28-f2d8-4ebd-8ee2-60920533184b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:55.900282 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d6b28-f2d8-4ebd-8ee2-60920533184b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:55.900551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d6b28-f2d8-4ebd-8ee2-60920533184b admin admin] PATH_INFO: `/v3/projects/86dd4d9a2c924076a121716be57bc442` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:55.901675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303d6b28-f2d8-4ebd-8ee2-60920533184b admin admin] RBAC: Authorizing `identity:delete_project(project_id=86dd4d9a2c924076a121716be57bc442)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:55.903780 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303d6b28-f2d8-4ebd-8ee2-60920533184b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:56.058496 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-303d6b28-f2d8-4ebd-8ee2-60920533184b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86dd4d9a2c924076a121716be57bc442', 'request_id': 'req-303d6b28-f2d8-4ebd-8ee2-60920533184b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:56.065019 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-303d6b28-f2d8-4ebd-8ee2-60920533184b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86dd4d9a2c924076a121716be57bc442', 'request_id': 'req-303d6b28-f2d8-4ebd-8ee2-60920533184b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:56.071914 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:56.073752 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-303d6b28-f2d8-4ebd-8ee2-60920533184b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86dd4d9a2c924076a121716be57bc442', 'request_id': 'req-303d6b28-f2d8-4ebd-8ee2-60920533184b'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:56.159470 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-303d6b28-f2d8-4ebd-8ee2-60920533184b admin admin] The token cache is being invalidate because project 86dd4d9a2c924076a121716be57bc442 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:03:56.159781 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-303d6b28-f2d8-4ebd-8ee2-60920533184b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:56.160475 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-303d6b28-f2d8-4ebd-8ee2-60920533184b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:56.162567 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1165/2350] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:03:55 2022] DELETE /identity/v3/projects/86dd4d9a2c924076a121716be57bc442 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:03:56.167121 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe82b6-8b77-4615-aae1-81062917a0b7 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:56.167585 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe82b6-8b77-4615-aae1-81062917a0b7 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:56.167721 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe82b6-8b77-4615-aae1-81062917a0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:56.187916 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-03fe82b6-8b77-4615-aae1-81062917a0b7 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:56.189091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-03fe82b6-8b77-4615-aae1-81062917a0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:56.250960 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1186/2351] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:56.259900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102f86a-36be-4183-8002-153bb1458aeb None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:56.260183 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102f86a-36be-4183-8002-153bb1458aeb None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:56.260457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102f86a-36be-4183-8002-153bb1458aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:56.280285 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a102f86a-36be-4183-8002-153bb1458aeb None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:56.281537 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a102f86a-36be-4183-8002-153bb1458aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:56.289619 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1166/2352] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:56.293990 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe82b6-8b77-4615-aae1-81062917a0b7 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:56.306360 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe82b6-8b77-4615-aae1-81062917a0b7 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:56.313711 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f027d47-344e-4929-bde1-b8044811627e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:56.314347 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f027d47-344e-4929-bde1-b8044811627e admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:56.314730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f027d47-344e-4929-bde1-b8044811627e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:56.314983 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f027d47-344e-4929-bde1-b8044811627e admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:56.315633 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f027d47-344e-4929-bde1-b8044811627e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:03:56.315922 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f027d47-344e-4929-bde1-b8044811627e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:56.317965 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f027d47-344e-4929-bde1-b8044811627e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:56.332010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1187/2353] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:03:56.336828 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102f86a-36be-4183-8002-153bb1458aeb None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:56.346355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102f86a-36be-4183-8002-153bb1458aeb None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:56.357870 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaee1ac-24b7-43e4-9a3a-58fefb71a47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:56.357870 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaee1ac-24b7-43e4-9a3a-58fefb71a47f admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:56.357870 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaee1ac-24b7-43e4-9a3a-58fefb71a47f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:56.357870 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaee1ac-24b7-43e4-9a3a-58fefb71a47f admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:56.358233 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaee1ac-24b7-43e4-9a3a-58fefb71a47f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:56.358233 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaee1ac-24b7-43e4-9a3a-58fefb71a47f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:56.491347 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1167/2354] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:03:56 2022] POST /identity/v3/projects => generated 365 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:03:56.495823 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f027d47-344e-4929-bde1-b8044811627e admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:56.505668 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f027d47-344e-4929-bde1-b8044811627e admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:56.557485 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:56.573151 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53120e93-a2f0-407e-9e45-9cfeb0573dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:56.573728 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53120e93-a2f0-407e-9e45-9cfeb0573dd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:56.574001 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53120e93-a2f0-407e-9e45-9cfeb0573dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:56.574313 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53120e93-a2f0-407e-9e45-9cfeb0573dd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:56.574934 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53120e93-a2f0-407e-9e45-9cfeb0573dd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:56.576827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53120e93-a2f0-407e-9e45-9cfeb0573dd4 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:56.670904 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1188/2355] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:03:56 2022] POST /identity/v3/users => generated 283 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:03:56.674620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaee1ac-24b7-43e4-9a3a-58fefb71a47f admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:56.687307 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaee1ac-24b7-43e4-9a3a-58fefb71a47f admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:56.694279 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c04a8-cdec-42da-87a4-0862f50b4b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:56.695147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c04a8-cdec-42da-87a4-0862f50b4b51 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:56.695435 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c04a8-cdec-42da-87a4-0862f50b4b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:56.695704 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c04a8-cdec-42da-87a4-0862f50b4b51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:56.696379 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c04a8-cdec-42da-87a4-0862f50b4b51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:56.698283 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c04a8-cdec-42da-87a4-0862f50b4b51 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:56.709041 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1168/2356] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:03:56.714216 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53120e93-a2f0-407e-9e45-9cfeb0573dd4 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:56.738618 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53120e93-a2f0-407e-9e45-9cfeb0573dd4 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:56.754046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74145556-27b8-41d3-bd38-51b3b95bd044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:56.754880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74145556-27b8-41d3-bd38-51b3b95bd044 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:56.755138 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74145556-27b8-41d3-bd38-51b3b95bd044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:56.755418 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74145556-27b8-41d3-bd38-51b3b95bd044 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:56.757154 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74145556-27b8-41d3-bd38-51b3b95bd044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:56.759067 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74145556-27b8-41d3-bd38-51b3b95bd044 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:56.774718 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1189/2357] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:03:56.776197 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c04a8-cdec-42da-87a4-0862f50b4b51 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:56.786886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c04a8-cdec-42da-87a4-0862f50b4b51 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:56.797581 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a85c7f-6a29-42ff-84be-6d41db2fdf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:56.798890 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a85c7f-6a29-42ff-84be-6d41db2fdf9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:56.798890 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a85c7f-6a29-42ff-84be-6d41db2fdf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:56.798890 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a85c7f-6a29-42ff-84be-6d41db2fdf9a admin admin] PATH_INFO: `/v3/projects/d074cc609d8c41f398d67b7e4d952bfa/users/a79bf6ab602a4d3184e9ea570dd4eeb3/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:56.820671 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a85c7f-6a29-42ff-84be-6d41db2fdf9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d074cc609d8c41f398d67b7e4d952bfa, user_id=a79bf6ab602a4d3184e9ea570dd4eeb3, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:56.824875 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a85c7f-6a29-42ff-84be-6d41db2fdf9a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:56.892066 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74145556-27b8-41d3-bd38-51b3b95bd044 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:56.893794 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-74145556-27b8-41d3-bd38-51b3b95bd044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:56.906888 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74145556-27b8-41d3-bd38-51b3b95bd044 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:56.911655 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a324acd-f34f-493c-963a-15f8a3738cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:56.913074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a324acd-f34f-493c-963a-15f8a3738cbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:56.913546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a324acd-f34f-493c-963a-15f8a3738cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:56.914007 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a324acd-f34f-493c-963a-15f8a3738cbd admin admin] PATH_INFO: `/v3/users/ffd55dbc683b4bcf8e9a3ae82e07cf3d` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:56.915171 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a324acd-f34f-493c-963a-15f8a3738cbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffd55dbc683b4bcf8e9a3ae82e07cf3d)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:56.917090 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a324acd-f34f-493c-963a-15f8a3738cbd admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:56.930501 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1169/2358] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:03:56 2022] PUT /identity/v3/projects/d074cc609d8c41f398d67b7e4d952bfa/users/a79bf6ab602a4d3184e9ea570dd4eeb3/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:03:56.943729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfd30c-145c-4e98-9a9d-e2c49a89acd0 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:56.944529 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfd30c-145c-4e98-9a9d-e2c49a89acd0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:56.944529 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfd30c-145c-4e98-9a9d-e2c49a89acd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:56.977486 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d3cfd30c-145c-4e98-9a9d-e2c49a89acd0 None None] MFA Rules not processed for user `a79bf6ab602a4d3184e9ea570dd4eeb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:56.978781 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d3cfd30c-145c-4e98-9a9d-e2c49a89acd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:57.047661 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1170/2359] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:03:57.062099 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfd30c-145c-4e98-9a9d-e2c49a89acd0 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:57.063534 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d3cfd30c-145c-4e98-9a9d-e2c49a89acd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:57.126347 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfd30c-145c-4e98-9a9d-e2c49a89acd0 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:57.129807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38994e58-a49b-4c18-9da3-77f86124d67d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:57.130556 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38994e58-a49b-4c18-9da3-77f86124d67d service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:57.131998 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38994e58-a49b-4c18-9da3-77f86124d67d service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:57.132267 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38994e58-a49b-4c18-9da3-77f86124d67d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:57.142878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38994e58-a49b-4c18-9da3-77f86124d67d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:57.144386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38994e58-a49b-4c18-9da3-77f86124d67d service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:57.153594 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1171/2360] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:03:57.166226 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0a324acd-f34f-493c-963a-15f8a3738cbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffd55dbc683b4bcf8e9a3ae82e07cf3d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:57.178867 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0a324acd-f34f-493c-963a-15f8a3738cbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffd55dbc683b4bcf8e9a3ae82e07cf3d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:57.197196 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0a324acd-f34f-493c-963a-15f8a3738cbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffd55dbc683b4bcf8e9a3ae82e07cf3d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:57.197611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0a324acd-f34f-493c-963a-15f8a3738cbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffd55dbc683b4bcf8e9a3ae82e07cf3d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:57.288640 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-0a324acd-f34f-493c-963a-15f8a3738cbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffd55dbc683b4bcf8e9a3ae82e07cf3d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:57.314133 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1190/2361] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:03:56 2022] DELETE /identity/v3/users/ffd55dbc683b4bcf8e9a3ae82e07cf3d => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:03:57.317490 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7f32c-77f4-4c59-8177-54b087f5acbb None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:57.317958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7f32c-77f4-4c59-8177-54b087f5acbb None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:57.318370 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7f32c-77f4-4c59-8177-54b087f5acbb None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:57.320437 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1172/2362] 10.222.0.17 () {54 vars in 921 bytes} [Sun Sep 11 00:03:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:03:57.322797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237381e-87e4-46c6-bfaf-68f29fb9cd60 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:57.323113 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237381e-87e4-46c6-bfaf-68f29fb9cd60 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:57.323494 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237381e-87e4-46c6-bfaf-68f29fb9cd60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:57.326902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76075d-78cc-49da-ac3b-9e552e11370a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:57.328504 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76075d-78cc-49da-ac3b-9e552e11370a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:57.328504 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76075d-78cc-49da-ac3b-9e552e11370a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:57.352536 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-3237381e-87e4-46c6-bfaf-68f29fb9cd60 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:57.352806 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0e76075d-78cc-49da-ac3b-9e552e11370a None None] MFA Rules not processed for user `636bfafb693044d283530063047fc0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:57.354012 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0e76075d-78cc-49da-ac3b-9e552e11370a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:57.354209 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3237381e-87e4-46c6-bfaf-68f29fb9cd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:57.371396 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:57.430251 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1191/2363] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:57.439871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237381e-87e4-46c6-bfaf-68f29fb9cd60 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:57.441389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3237381e-87e4-46c6-bfaf-68f29fb9cd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:57.453597 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1173/2364] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Sep 11 00:03:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:03:57.468692 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76075d-78cc-49da-ac3b-9e552e11370a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:57.470226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0e76075d-78cc-49da-ac3b-9e552e11370a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:57.485690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:03:57.512704 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237381e-87e4-46c6-bfaf-68f29fb9cd60 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:57.514704 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7dc698-7769-4463-ac8d-faf77ff4ff98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:57.515442 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7dc698-7769-4463-ac8d-faf77ff4ff98 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:57.515715 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7dc698-7769-4463-ac8d-faf77ff4ff98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:57.515988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7dc698-7769-4463-ac8d-faf77ff4ff98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:57.527599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7dc698-7769-4463-ac8d-faf77ff4ff98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:57.529051 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7dc698-7769-4463-ac8d-faf77ff4ff98 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:57.531390 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76075d-78cc-49da-ac3b-9e552e11370a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:57.535680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee77b84-08cb-4975-b843-6b502c69dee4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb18199dc8b42589184042556e98983', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:57.536276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee77b84-08cb-4975-b843-6b502c69dee4 service glance] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:57.536554 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee77b84-08cb-4975-b843-6b502c69dee4 service glance] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:57.536824 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee77b84-08cb-4975-b843-6b502c69dee4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:57.538759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1192/2365] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:03:57.548263 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee77b84-08cb-4975-b843-6b502c69dee4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:57.549726 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee77b84-08cb-4975-b843-6b502c69dee4 service glance] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:57.560732 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1174/2366] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:03:57.800723 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951199a-4fef-43a1-9782-cd5cd66137de None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:57.801168 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951199a-4fef-43a1-9782-cd5cd66137de None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:57.801539 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951199a-4fef-43a1-9782-cd5cd66137de None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:57.803903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1193/2367] 10.222.0.17 () {54 vars in 921 bytes} [Sun Sep 11 00:03:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:03:57.810139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca6e2e-47af-45ab-bf6c-63e88edeeb46 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:57.810455 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca6e2e-47af-45ab-bf6c-63e88edeeb46 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:57.810836 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca6e2e-47af-45ab-bf6c-63e88edeeb46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:57.833658 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-32ca6e2e-47af-45ab-bf6c-63e88edeeb46 None None] MFA Rules not processed for user `53e79d9fb2cd452d8d999a84b6425959`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:57.835013 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-32ca6e2e-47af-45ab-bf6c-63e88edeeb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:57.843157 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1175/2368] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Sep 11 00:03:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:03:57.858804 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951199a-4fef-43a1-9782-cd5cd66137de None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:57.870971 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951199a-4fef-43a1-9782-cd5cd66137de None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:57.883404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71372302-3996-4c55-981d-28043609fdf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:57.885191 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71372302-3996-4c55-981d-28043609fdf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:57.885450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71372302-3996-4c55-981d-28043609fdf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:57.885697 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71372302-3996-4c55-981d-28043609fdf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:57.908146 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71372302-3996-4c55-981d-28043609fdf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:57.909931 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71372302-3996-4c55-981d-28043609fdf3 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:57.920262 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1194/2369] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:03:58.026014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca6e2e-47af-45ab-bf6c-63e88edeeb46 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:58.027719 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-32ca6e2e-47af-45ab-bf6c-63e88edeeb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:58.044589 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca6e2e-47af-45ab-bf6c-63e88edeeb46 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:58.058119 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d418c-b7fc-4bf2-a89f-629ec7bd88b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:58.058993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d418c-b7fc-4bf2-a89f-629ec7bd88b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:58.059386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d418c-b7fc-4bf2-a89f-629ec7bd88b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:58.059720 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d418c-b7fc-4bf2-a89f-629ec7bd88b3 admin admin] PATH_INFO: `/v3/projects/8f2e66e3f8ee4e85960d55afde84ef7a` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:58.061606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d418c-b7fc-4bf2-a89f-629ec7bd88b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f2e66e3f8ee4e85960d55afde84ef7a)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:58.063665 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d418c-b7fc-4bf2-a89f-629ec7bd88b3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:58.099572 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28b2d3-7baa-4862-a69b-170620f820b0 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:58.100407 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28b2d3-7baa-4862-a69b-170620f820b0 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:58.100678 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28b2d3-7baa-4862-a69b-170620f820b0 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:58.102811 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1195/2370] 10.222.0.17 () {54 vars in 921 bytes} [Sun Sep 11 00:03:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 00:03:58.108779 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71569ac-3b82-4081-9622-ec4746820e99 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:58.109134 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71569ac-3b82-4081-9622-ec4746820e99 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:58.109478 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71569ac-3b82-4081-9622-ec4746820e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:58.142320 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-d71569ac-3b82-4081-9622-ec4746820e99 None None] MFA Rules not processed for user `636bfafb693044d283530063047fc0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:03:58.143651 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-d71569ac-3b82-4081-9622-ec4746820e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:03:58.159066 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1196/2371] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Sep 11 00:03:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:03:58.171447 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71569ac-3b82-4081-9622-ec4746820e99 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:03:58.183209 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71569ac-3b82-4081-9622-ec4746820e99 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:03:58.187015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd79a3e-dd2f-43f1-a88b-e99d6ad3d880 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:03:58.187692 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd79a3e-dd2f-43f1-a88b-e99d6ad3d880 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:03:58.188063 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd79a3e-dd2f-43f1-a88b-e99d6ad3d880 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:03:58.188395 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd79a3e-dd2f-43f1-a88b-e99d6ad3d880 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:03:58.203365 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd79a3e-dd2f-43f1-a88b-e99d6ad3d880 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:03:58.204920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd79a3e-dd2f-43f1-a88b-e99d6ad3d880 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:03:58.218287 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1197/2372] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:03:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:03:58.219257 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-902d418c-b7fc-4bf2-a89f-629ec7bd88b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f2e66e3f8ee4e85960d55afde84ef7a', 'request_id': 'req-902d418c-b7fc-4bf2-a89f-629ec7bd88b3'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:58.226924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-902d418c-b7fc-4bf2-a89f-629ec7bd88b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f2e66e3f8ee4e85960d55afde84ef7a', 'request_id': 'req-902d418c-b7fc-4bf2-a89f-629ec7bd88b3'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:58.232871 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-902d418c-b7fc-4bf2-a89f-629ec7bd88b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f2e66e3f8ee4e85960d55afde84ef7a', 'request_id': 'req-902d418c-b7fc-4bf2-a89f-629ec7bd88b3'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:58.328989 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-902d418c-b7fc-4bf2-a89f-629ec7bd88b3 admin admin] The token cache is being invalidate because project 8f2e66e3f8ee4e85960d55afde84ef7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:03:58.329608 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-902d418c-b7fc-4bf2-a89f-629ec7bd88b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:58.330503 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-902d418c-b7fc-4bf2-a89f-629ec7bd88b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:03:58.332269 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1176/2373] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:03:58 2022] DELETE /identity/v3/projects/8f2e66e3f8ee4e85960d55afde84ef7a => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:04:03.672968 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84f836-d505-4387-94e5-4dd7872cebba None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:04:03.673423 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84f836-d505-4387-94e5-4dd7872cebba None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:04:03.673511 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84f836-d505-4387-94e5-4dd7872cebba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:04:03.692182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-8e84f836-d505-4387-94e5-4dd7872cebba None None] MFA Rules not processed for user `b0a879085a194b0687e1cdcbafd48033`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:04:03.693614 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8e84f836-d505-4387-94e5-4dd7872cebba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:04:03.751583 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1198/2374] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Sep 11 00:04:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:04:03.764594 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d418c-b7fc-4bf2-a89f-629ec7bd88b3 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:04:03.766000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-902d418c-b7fc-4bf2-a89f-629ec7bd88b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:04:03.804168 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:04:03.845366 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d418c-b7fc-4bf2-a89f-629ec7bd88b3 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:04:03.850283 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952dc75d-b8f6-4904-a36f-2c6568cbe8e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '564e65585e9543e286b6041af870cf41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:04:03.850915 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952dc75d-b8f6-4904-a36f-2c6568cbe8e0 service swift] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:04:03.852055 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952dc75d-b8f6-4904-a36f-2c6568cbe8e0 service swift] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:04:03.852542 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952dc75d-b8f6-4904-a36f-2c6568cbe8e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:04:03.863028 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952dc75d-b8f6-4904-a36f-2c6568cbe8e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:04:03.864762 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952dc75d-b8f6-4904-a36f-2c6568cbe8e0 service swift] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:04:03.875180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1177/2375] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Sep 11 00:04:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 00:04:13.745460 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=50 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:04:13.897025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=19 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:04:31.849697 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62e75d-ecec-4c52-bd91-2c071518ced9 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:04:31.850217 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62e75d-ecec-4c52-bd91-2c071518ced9 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:04:31.850217 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62e75d-ecec-4c52-bd91-2c071518ced9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:04:31.869598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-2d62e75d-ecec-4c52-bd91-2c071518ced9 None None] MFA Rules not processed for user `b0a879085a194b0687e1cdcbafd48033`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:04:31.870821 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2d62e75d-ecec-4c52-bd91-2c071518ced9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:04:31.877835 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1199/2376] 10.222.0.17 () {58 vars in 967 bytes} [Sun Sep 11 00:04:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:04:31.900302 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952dc75d-b8f6-4904-a36f-2c6568cbe8e0 service swift] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:04:31.911323 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952dc75d-b8f6-4904-a36f-2c6568cbe8e0 service swift] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:04:31.916675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0ac08-651a-4e9a-9134-7d5ccccb9acd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '564e65585e9543e286b6041af870cf41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:04:31.917269 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0ac08-651a-4e9a-9134-7d5ccccb9acd service swift] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:04:31.917569 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0ac08-651a-4e9a-9134-7d5ccccb9acd service swift] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:04:31.917941 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0ac08-651a-4e9a-9134-7d5ccccb9acd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:04:31.925963 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0ac08-651a-4e9a-9134-7d5ccccb9acd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:04:31.927571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0ac08-651a-4e9a-9134-7d5ccccb9acd service swift] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:04:31.935673 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1178/2377] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Sep 11 00:04:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 00:04:41.864035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:04:41.932518 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=6 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:04:44.903715 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62e75d-ecec-4c52-bd91-2c071518ced9 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:04:44.905544 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2d62e75d-ecec-4c52-bd91-2c071518ced9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:04:44.980788 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62e75d-ecec-4c52-bd91-2c071518ced9 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:04:44.985437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12ec58-56bb-4de8-87ae-4259edaedfa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:04:44.986098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12ec58-56bb-4de8-87ae-4259edaedfa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:04:44.986417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12ec58-56bb-4de8-87ae-4259edaedfa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:04:44.986717 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12ec58-56bb-4de8-87ae-4259edaedfa8 admin admin] PATH_INFO: `/v3/users/a79bf6ab602a4d3184e9ea570dd4eeb3` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:04:44.992111 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed12ec58-56bb-4de8-87ae-4259edaedfa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a79bf6ab602a4d3184e9ea570dd4eeb3)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:04:44.992111 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed12ec58-56bb-4de8-87ae-4259edaedfa8 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:04:45.224402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ed12ec58-56bb-4de8-87ae-4259edaedfa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a79bf6ab602a4d3184e9ea570dd4eeb3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:04:45.237819 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ed12ec58-56bb-4de8-87ae-4259edaedfa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a79bf6ab602a4d3184e9ea570dd4eeb3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:04:45.244892 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ed12ec58-56bb-4de8-87ae-4259edaedfa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a79bf6ab602a4d3184e9ea570dd4eeb3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:04:45.248450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ed12ec58-56bb-4de8-87ae-4259edaedfa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a79bf6ab602a4d3184e9ea570dd4eeb3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:04:45.328869 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-ed12ec58-56bb-4de8-87ae-4259edaedfa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a79bf6ab602a4d3184e9ea570dd4eeb3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:04:45.335309 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1200/2378] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:04:44 2022] DELETE /identity/v3/users/a79bf6ab602a4d3184e9ea570dd4eeb3 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:04:45.340575 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012022f1-3a31-4f42-9f79-f651fbd0f239 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:04:45.341016 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012022f1-3a31-4f42-9f79-f651fbd0f239 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:04:45.341117 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012022f1-3a31-4f42-9f79-f651fbd0f239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:04:45.359771 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-012022f1-3a31-4f42-9f79-f651fbd0f239 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:04:45.360944 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-012022f1-3a31-4f42-9f79-f651fbd0f239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:04:45.422899 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1179/2379] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:04:45.432937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12ec58-56bb-4de8-87ae-4259edaedfa8 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:04:45.434384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ed12ec58-56bb-4de8-87ae-4259edaedfa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:04:45.490678 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12ec58-56bb-4de8-87ae-4259edaedfa8 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:04:45.494118 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358e6d0-bf41-4d9d-8d9a-2cf803b7cafc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:04:45.494742 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358e6d0-bf41-4d9d-8d9a-2cf803b7cafc service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:04:45.495014 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358e6d0-bf41-4d9d-8d9a-2cf803b7cafc service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:04:45.495280 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358e6d0-bf41-4d9d-8d9a-2cf803b7cafc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:04:45.503957 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358e6d0-bf41-4d9d-8d9a-2cf803b7cafc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:04:45.505366 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358e6d0-bf41-4d9d-8d9a-2cf803b7cafc service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:04:45.514987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1201/2380] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:04:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:04:45.902807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012022f1-3a31-4f42-9f79-f651fbd0f239 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:04:45.904337 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-012022f1-3a31-4f42-9f79-f651fbd0f239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:04:45.916483 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012022f1-3a31-4f42-9f79-f651fbd0f239 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:04:45.921363 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b7932-8e26-4d76-9746-fa41b53486c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:04:45.922036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b7932-8e26-4d76-9746-fa41b53486c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:04:45.922344 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b7932-8e26-4d76-9746-fa41b53486c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:04:45.922609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b7932-8e26-4d76-9746-fa41b53486c6 admin admin] PATH_INFO: `/v3/projects/d074cc609d8c41f398d67b7e4d952bfa` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:04:45.923856 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b7932-8e26-4d76-9746-fa41b53486c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d074cc609d8c41f398d67b7e4d952bfa)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:04:45.925810 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b7932-8e26-4d76-9746-fa41b53486c6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:04:46.062113 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b75b7932-8e26-4d76-9746-fa41b53486c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd074cc609d8c41f398d67b7e4d952bfa', 'request_id': 'req-b75b7932-8e26-4d76-9746-fa41b53486c6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:04:46.068553 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b75b7932-8e26-4d76-9746-fa41b53486c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd074cc609d8c41f398d67b7e4d952bfa', 'request_id': 'req-b75b7932-8e26-4d76-9746-fa41b53486c6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:04:46.075472 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b75b7932-8e26-4d76-9746-fa41b53486c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd074cc609d8c41f398d67b7e4d952bfa', 'request_id': 'req-b75b7932-8e26-4d76-9746-fa41b53486c6'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:04:46.162600 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b75b7932-8e26-4d76-9746-fa41b53486c6 admin admin] The token cache is being invalidate because project d074cc609d8c41f398d67b7e4d952bfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:04:46.162857 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b75b7932-8e26-4d76-9746-fa41b53486c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:04:46.163510 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b75b7932-8e26-4d76-9746-fa41b53486c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:04:46.165013 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1180/2381] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:04:45 2022] DELETE /identity/v3/projects/d074cc609d8c41f398d67b7e4d952bfa => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:04:50.835536 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358e6d0-bf41-4d9d-8d9a-2cf803b7cafc service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:04:50.837004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4358e6d0-bf41-4d9d-8d9a-2cf803b7cafc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:04:50.866323 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:04:50.909783 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358e6d0-bf41-4d9d-8d9a-2cf803b7cafc service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:04:50.971267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a231a7-d2bc-4e40-879c-74501c14adce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cb1d85a338e40f18e61887a1becb906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:04:50.971914 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a231a7-d2bc-4e40-879c-74501c14adce service placement] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:04:50.972226 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a231a7-d2bc-4e40-879c-74501c14adce service placement] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:04:50.972490 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a231a7-d2bc-4e40-879c-74501c14adce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:04:50.974215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f6a231a7-d2bc-4e40-879c-74501c14adce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:04:50.985761 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a231a7-d2bc-4e40-879c-74501c14adce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:04:50.990960 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a231a7-d2bc-4e40-879c-74501c14adce service placement] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:04:50.997720 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1202/2382] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:04:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:04:56.159515 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:05:01.009043 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=48 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:05:02.731717 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c64bd-a8c8-469b-9f04-3c7b90b47c5c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:05:02.732211 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c64bd-a8c8-469b-9f04-3c7b90b47c5c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:05:02.732316 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c64bd-a8c8-469b-9f04-3c7b90b47c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:05:02.753415 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b78c64bd-a8c8-469b-9f04-3c7b90b47c5c None None] MFA Rules not processed for user `017684196c8e49e0894d38daf83bfaeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:05:02.754637 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b78c64bd-a8c8-469b-9f04-3c7b90b47c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:05:02.810516 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1181/2383] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:05:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:05:12.802855 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=25 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:05:36.285992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad108e6b-fa21-46aa-957a-afc1839240c4 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:05:36.286515 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad108e6b-fa21-46aa-957a-afc1839240c4 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:05:36.286515 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad108e6b-fa21-46aa-957a-afc1839240c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:05:36.306369 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-ad108e6b-fa21-46aa-957a-afc1839240c4 None None] MFA Rules not processed for user `b8ceeb13c4c24ceb9ff1f27b4e45f62c`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:05:36.307597 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ad108e6b-fa21-46aa-957a-afc1839240c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:05:36.364471 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1203/2384] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:05:36 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:05:45.281866 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88459f8-4d83-4a76-ad40-44efea310bc8 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:05:45.282448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88459f8-4d83-4a76-ad40-44efea310bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:05:45.282448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88459f8-4d83-4a76-ad40-44efea310bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:05:45.303257 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d88459f8-4d83-4a76-ad40-44efea310bc8 None None] MFA Rules not processed for user `a66188ce11bc403d9c657efd78f3019a`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:05:45.304539 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d88459f8-4d83-4a76-ad40-44efea310bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:05:45.366948 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1182/2385] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:05:45 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:05:46.356937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=25 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:05:55.359622 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=25 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:05:56.685839 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad108e6b-fa21-46aa-957a-afc1839240c4 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:05:56.687418 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ad108e6b-fa21-46aa-957a-afc1839240c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:05:56.749229 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad108e6b-fa21-46aa-957a-afc1839240c4 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:05:56.753313 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a995bbe-bdcb-4971-a6b7-0b9a544a6be5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:05:56.753918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a995bbe-bdcb-4971-a6b7-0b9a544a6be5 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:05:56.754190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a995bbe-bdcb-4971-a6b7-0b9a544a6be5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:05:56.754456 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a995bbe-bdcb-4971-a6b7-0b9a544a6be5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:05:56.756196 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1a995bbe-bdcb-4971-a6b7-0b9a544a6be5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:05:56.837300 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a995bbe-bdcb-4971-a6b7-0b9a544a6be5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:05:56.838730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a995bbe-bdcb-4971-a6b7-0b9a544a6be5 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:05:56.850329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1204/2386] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:05:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:05:58.046940 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88459f8-4d83-4a76-ad40-44efea310bc8 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:05:58.048394 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d88459f8-4d83-4a76-ad40-44efea310bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:05:58.060059 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88459f8-4d83-4a76-ad40-44efea310bc8 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:05:58.063637 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c9da6-1e8a-4cb4-972f-4cfb463e6bef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:05:58.064585 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c9da6-1e8a-4cb4-972f-4cfb463e6bef service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:05:58.064585 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c9da6-1e8a-4cb4-972f-4cfb463e6bef service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:05:58.064881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c9da6-1e8a-4cb4-972f-4cfb463e6bef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:05:58.066755 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-393c9da6-1e8a-4cb4-972f-4cfb463e6bef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:05:58.080408 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c9da6-1e8a-4cb4-972f-4cfb463e6bef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:05:58.082477 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c9da6-1e8a-4cb4-972f-4cfb463e6bef service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:05:58.093950 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1183/2387] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:05:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:06:06.844722 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=54 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:06:08.088843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=6 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:06:08.819236 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e05972-1ef6-4401-b1f0-a62af2714947 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:08.819752 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e05972-1ef6-4401-b1f0-a62af2714947 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:08.819849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e05972-1ef6-4401-b1f0-a62af2714947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:08.839788 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-c3e05972-1ef6-4401-b1f0-a62af2714947 None None] MFA Rules not processed for user `9950f578162449c0845c3aa4aab692d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:06:08.840978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c3e05972-1ef6-4401-b1f0-a62af2714947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:06:08.896501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1205/2388] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:06:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:06:08.908034 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c9da6-1e8a-4cb4-972f-4cfb463e6bef service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:06:08.909775 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-393c9da6-1e8a-4cb4-972f-4cfb463e6bef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:06:08.923034 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c9da6-1e8a-4cb4-972f-4cfb463e6bef service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:06:08.928684 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78e8cf-d2e1-4de9-a73e-41df42cbae5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:06:08.929265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78e8cf-d2e1-4de9-a73e-41df42cbae5e service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:08.929542 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78e8cf-d2e1-4de9-a73e-41df42cbae5e service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:08.929811 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78e8cf-d2e1-4de9-a73e-41df42cbae5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:08.931476 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3b78e8cf-d2e1-4de9-a73e-41df42cbae5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:06:08.942078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b78e8cf-d2e1-4de9-a73e-41df42cbae5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:06:08.943670 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b78e8cf-d2e1-4de9-a73e-41df42cbae5e service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:06:08.952549 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1184/2389] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:06:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:06:14.057193 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e05972-1ef6-4401-b1f0-a62af2714947 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:06:14.058668 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c3e05972-1ef6-4401-b1f0-a62af2714947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:06:14.129785 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e05972-1ef6-4401-b1f0-a62af2714947 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:06:14.135290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580a6cd-dca2-42e3-a6e7-aa4b5321e983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:06:14.136147 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580a6cd-dca2-42e3-a6e7-aa4b5321e983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:14.136427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580a6cd-dca2-42e3-a6e7-aa4b5321e983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:14.136700 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580a6cd-dca2-42e3-a6e7-aa4b5321e983 admin admin] PATH_INFO: `/v3/users/017684196c8e49e0894d38daf83bfaeb` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:14.137836 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580a6cd-dca2-42e3-a6e7-aa4b5321e983 admin admin] RBAC: Authorizing `identity:delete_user(user_id=017684196c8e49e0894d38daf83bfaeb)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:06:14.139813 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580a6cd-dca2-42e3-a6e7-aa4b5321e983 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:06:14.302633 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c580a6cd-dca2-42e3-a6e7-aa4b5321e983 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '017684196c8e49e0894d38daf83bfaeb'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:06:14.320244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c580a6cd-dca2-42e3-a6e7-aa4b5321e983 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '017684196c8e49e0894d38daf83bfaeb'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:06:14.329697 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c580a6cd-dca2-42e3-a6e7-aa4b5321e983 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '017684196c8e49e0894d38daf83bfaeb'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:06:14.330954 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c580a6cd-dca2-42e3-a6e7-aa4b5321e983 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '017684196c8e49e0894d38daf83bfaeb'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:06:14.392699 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c580a6cd-dca2-42e3-a6e7-aa4b5321e983 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '017684196c8e49e0894d38daf83bfaeb'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:06:14.402491 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1206/2390] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:06:14 2022] DELETE /identity/v3/users/017684196c8e49e0894d38daf83bfaeb => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:06:14.683346 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78e8cf-d2e1-4de9-a73e-41df42cbae5e service nova] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:06:14.694992 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78e8cf-d2e1-4de9-a73e-41df42cbae5e service nova] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:06:14.752790 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f73f2-2bbe-4621-bd51-ba6d1343a011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:06:14.753443 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f73f2-2bbe-4621-bd51-ba6d1343a011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:14.753724 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f73f2-2bbe-4621-bd51-ba6d1343a011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:14.754194 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f73f2-2bbe-4621-bd51-ba6d1343a011 admin admin] PATH_INFO: `/v3/projects/fa03f6136eb640ad803659779ce1b085` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:14.755141 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f73f2-2bbe-4621-bd51-ba6d1343a011 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa03f6136eb640ad803659779ce1b085)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:06:14.757055 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f73f2-2bbe-4621-bd51-ba6d1343a011 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:06:14.878000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-af8f73f2-2bbe-4621-bd51-ba6d1343a011 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa03f6136eb640ad803659779ce1b085', 'request_id': 'req-af8f73f2-2bbe-4621-bd51-ba6d1343a011'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:06:14.884372 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-af8f73f2-2bbe-4621-bd51-ba6d1343a011 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa03f6136eb640ad803659779ce1b085', 'request_id': 'req-af8f73f2-2bbe-4621-bd51-ba6d1343a011'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:06:14.889891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-af8f73f2-2bbe-4621-bd51-ba6d1343a011 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa03f6136eb640ad803659779ce1b085', 'request_id': 'req-af8f73f2-2bbe-4621-bd51-ba6d1343a011'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:06:14.963361 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-af8f73f2-2bbe-4621-bd51-ba6d1343a011 admin admin] The token cache is being invalidate because project fa03f6136eb640ad803659779ce1b085 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:06:14.963561 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-af8f73f2-2bbe-4621-bd51-ba6d1343a011 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:06:14.964205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-af8f73f2-2bbe-4621-bd51-ba6d1343a011 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:06:14.965660 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1185/2391] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:06:14 2022] DELETE /identity/v3/projects/fa03f6136eb640ad803659779ce1b085 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:06:14.970759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bd879-c195-4ad0-8010-d6830ee717d9 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:14.971062 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bd879-c195-4ad0-8010-d6830ee717d9 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:14.971330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bd879-c195-4ad0-8010-d6830ee717d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:14.991403 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-e54bd879-c195-4ad0-8010-d6830ee717d9 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:06:14.992587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e54bd879-c195-4ad0-8010-d6830ee717d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:06:15.011186 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:06:15.052217 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1207/2392] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:06:15.060276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f95acd-5a68-4447-befe-00971f62ef86 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:15.060565 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f95acd-5a68-4447-befe-00971f62ef86 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:15.060896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f95acd-5a68-4447-befe-00971f62ef86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:15.081511 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-21f95acd-5a68-4447-befe-00971f62ef86 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:06:15.082820 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-21f95acd-5a68-4447-befe-00971f62ef86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:06:15.090573 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1186/2393] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:06:15.095084 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54bd879-c195-4ad0-8010-d6830ee717d9 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:06:15.105290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54bd879-c195-4ad0-8010-d6830ee717d9 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:06:15.109762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b93bc-f63a-4535-8362-4eae233b9039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:06:15.110353 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b93bc-f63a-4535-8362-4eae233b9039 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:15.110628 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b93bc-f63a-4535-8362-4eae233b9039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:15.110924 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b93bc-f63a-4535-8362-4eae233b9039 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:15.111558 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b93bc-f63a-4535-8362-4eae233b9039 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:06:15.111853 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b93bc-f63a-4535-8362-4eae233b9039 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:06:15.114244 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b93bc-f63a-4535-8362-4eae233b9039 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:06:15.125692 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1208/2394] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:06:15.130433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f95acd-5a68-4447-befe-00971f62ef86 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:06:15.139446 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f95acd-5a68-4447-befe-00971f62ef86 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:06:15.144649 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84827c-6e5d-49fd-afd3-3a79963545f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:06:15.145400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84827c-6e5d-49fd-afd3-3a79963545f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:15.145676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84827c-6e5d-49fd-afd3-3a79963545f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:15.145946 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84827c-6e5d-49fd-afd3-3a79963545f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:15.146563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee84827c-6e5d-49fd-afd3-3a79963545f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:06:15.148476 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee84827c-6e5d-49fd-afd3-3a79963545f9 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:06:15.212549 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1187/2395] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:06:15 2022] POST /identity/v3/projects => generated 383 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:06:15.222118 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b93bc-f63a-4535-8362-4eae233b9039 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:06:15.227015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b93bc-f63a-4535-8362-4eae233b9039 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:06:15.283582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cca3e7-2138-4aab-8970-34c44bd54bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:06:15.284179 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cca3e7-2138-4aab-8970-34c44bd54bff admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:15.284456 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cca3e7-2138-4aab-8970-34c44bd54bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:15.285079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cca3e7-2138-4aab-8970-34c44bd54bff admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:15.285429 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cca3e7-2138-4aab-8970-34c44bd54bff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:06:15.287526 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cca3e7-2138-4aab-8970-34c44bd54bff admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:06:15.382431 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1209/2396] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:06:15 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:06:15.386752 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84827c-6e5d-49fd-afd3-3a79963545f9 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:06:15.397529 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84827c-6e5d-49fd-afd3-3a79963545f9 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:06:15.402150 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b3cb2-42b4-4b84-9143-bca1b4751416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:06:15.402754 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b3cb2-42b4-4b84-9143-bca1b4751416 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:15.403026 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b3cb2-42b4-4b84-9143-bca1b4751416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:15.403291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b3cb2-42b4-4b84-9143-bca1b4751416 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:15.403918 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b3cb2-42b4-4b84-9143-bca1b4751416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:06:15.405500 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b3cb2-42b4-4b84-9143-bca1b4751416 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:06:15.415223 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1188/2397] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:06:15.419120 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cca3e7-2138-4aab-8970-34c44bd54bff admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:06:15.429657 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cca3e7-2138-4aab-8970-34c44bd54bff admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:06:15.434249 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b04fb1-2356-43e5-a5ed-0af8160a00b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:06:15.434877 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b04fb1-2356-43e5-a5ed-0af8160a00b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:15.435156 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b04fb1-2356-43e5-a5ed-0af8160a00b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:15.435424 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b04fb1-2356-43e5-a5ed-0af8160a00b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:15.436052 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b04fb1-2356-43e5-a5ed-0af8160a00b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:06:15.437651 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b04fb1-2356-43e5-a5ed-0af8160a00b5 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:06:15.452723 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1210/2398] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:06:15.457775 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b3cb2-42b4-4b84-9143-bca1b4751416 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:06:15.467746 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b3cb2-42b4-4b84-9143-bca1b4751416 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:06:15.472585 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c5526-3cf0-4688-9df6-3c3a963fe916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:06:15.473160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c5526-3cf0-4688-9df6-3c3a963fe916 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:15.473433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c5526-3cf0-4688-9df6-3c3a963fe916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:15.473703 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c5526-3cf0-4688-9df6-3c3a963fe916 admin admin] PATH_INFO: `/v3/projects/82b6b05c598d44ee8b8748b6bab1f9e5/users/fd106d45a1ce4aad9c85c35d4f53af92/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:15.488658 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c5526-3cf0-4688-9df6-3c3a963fe916 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82b6b05c598d44ee8b8748b6bab1f9e5, user_id=fd106d45a1ce4aad9c85c35d4f53af92, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:06:15.491635 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c5526-3cf0-4688-9df6-3c3a963fe916 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:06:15.584321 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1189/2399] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:06:15 2022] PUT /identity/v3/projects/82b6b05c598d44ee8b8748b6bab1f9e5/users/fd106d45a1ce4aad9c85c35d4f53af92/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:06:15.589070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dda58b-023e-46a0-9858-f574a0fa0163 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:15.589345 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dda58b-023e-46a0-9858-f574a0fa0163 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:15.589633 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dda58b-023e-46a0-9858-f574a0fa0163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:15.609124 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-62dda58b-023e-46a0-9858-f574a0fa0163 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:06:15.610361 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-62dda58b-023e-46a0-9858-f574a0fa0163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:06:15.674305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1211/2400] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:06:15.684394 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c5526-3cf0-4688-9df6-3c3a963fe916 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:06:15.686102 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-938c5526-3cf0-4688-9df6-3c3a963fe916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:06:15.701247 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:06:15.744257 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c5526-3cf0-4688-9df6-3c3a963fe916 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:06:15.747762 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63b526-d866-44a7-a6f0-d31598d99ae8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:06:15.748349 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63b526-d866-44a7-a6f0-d31598d99ae8 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:15.748627 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63b526-d866-44a7-a6f0-d31598d99ae8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:15.748894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63b526-d866-44a7-a6f0-d31598d99ae8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:15.758550 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63b526-d866-44a7-a6f0-d31598d99ae8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:06:15.760327 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63b526-d866-44a7-a6f0-d31598d99ae8 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:06:15.770354 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1190/2401] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:06:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:06:21.386539 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e42dac-7701-4eb2-934a-1ad74b44e767 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:21.387070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e42dac-7701-4eb2-934a-1ad74b44e767 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:21.387292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e42dac-7701-4eb2-934a-1ad74b44e767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:21.420887 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-33e42dac-7701-4eb2-934a-1ad74b44e767 None None] MFA Rules not processed for user `fd106d45a1ce4aad9c85c35d4f53af92`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:06:21.426393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-33e42dac-7701-4eb2-934a-1ad74b44e767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:06:21.447232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:06:21.501434 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1212/2402] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:06:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:06:21.506213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63b526-d866-44a7-a6f0-d31598d99ae8 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:06:21.516251 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63b526-d866-44a7-a6f0-d31598d99ae8 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:06:21.521788 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c7a42-33fa-427d-abec-3bba9934d638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:06:21.522558 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c7a42-33fa-427d-abec-3bba9934d638 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:21.524420 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c7a42-33fa-427d-abec-3bba9934d638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:21.524420 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c7a42-33fa-427d-abec-3bba9934d638 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:21.524420 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7c7a42-33fa-427d-abec-3bba9934d638 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:06:21.525740 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7c7a42-33fa-427d-abec-3bba9934d638 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:06:21.636120 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1191/2403] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:06:21 2022] POST /identity/v3/projects => generated 381 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:06:21.638894 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e42dac-7701-4eb2-934a-1ad74b44e767 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:06:21.650281 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e42dac-7701-4eb2-934a-1ad74b44e767 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:06:21.712872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df2a1f7-2582-44c9-8326-2e0821ed3789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:06:21.713479 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2a1f7-2582-44c9-8326-2e0821ed3789 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:21.713756 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2a1f7-2582-44c9-8326-2e0821ed3789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:21.714030 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2a1f7-2582-44c9-8326-2e0821ed3789 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:21.714684 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df2a1f7-2582-44c9-8326-2e0821ed3789 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:06:21.716741 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df2a1f7-2582-44c9-8326-2e0821ed3789 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:06:21.812494 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1213/2404] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:06:21 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:06:21.813285 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c7a42-33fa-427d-abec-3bba9934d638 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:06:21.825310 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c7a42-33fa-427d-abec-3bba9934d638 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:06:21.830563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0af48-6fb4-47af-a11a-34ecc51948d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:06:21.831288 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0af48-6fb4-47af-a11a-34ecc51948d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:21.837547 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0af48-6fb4-47af-a11a-34ecc51948d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:21.837547 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0af48-6fb4-47af-a11a-34ecc51948d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:21.837547 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e0af48-6fb4-47af-a11a-34ecc51948d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:06:21.837547 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e0af48-6fb4-47af-a11a-34ecc51948d4 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:06:21.845993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1192/2405] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:06:21.850476 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df2a1f7-2582-44c9-8326-2e0821ed3789 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:06:21.860691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df2a1f7-2582-44c9-8326-2e0821ed3789 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:06:21.865864 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f881e7d-5bbc-4861-8881-e663bc3f49ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:06:21.866838 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f881e7d-5bbc-4861-8881-e663bc3f49ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:21.867312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f881e7d-5bbc-4861-8881-e663bc3f49ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:21.868022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f881e7d-5bbc-4861-8881-e663bc3f49ac admin admin] PATH_INFO: `/v3/projects/cc0ef4d1c1a746e3a2467f2d852553f8/users/19fcb43f28984cd393dc9d9e28ac7c64/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:21.884533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f881e7d-5bbc-4861-8881-e663bc3f49ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc0ef4d1c1a746e3a2467f2d852553f8, user_id=19fcb43f28984cd393dc9d9e28ac7c64, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:06:21.887463 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f881e7d-5bbc-4861-8881-e663bc3f49ac admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:06:21.950273 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1214/2406] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:06:21 2022] PUT /identity/v3/projects/cc0ef4d1c1a746e3a2467f2d852553f8/users/19fcb43f28984cd393dc9d9e28ac7c64/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:06:21.954440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0af48-6fb4-47af-a11a-34ecc51948d4 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:06:21.965112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0af48-6fb4-47af-a11a-34ecc51948d4 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:06:22.022524 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6da62-dc86-4d92-ade3-a8e9a19950e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:06:22.023163 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6da62-dc86-4d92-ade3-a8e9a19950e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:22.023437 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6da62-dc86-4d92-ade3-a8e9a19950e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:22.023790 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6da62-dc86-4d92-ade3-a8e9a19950e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:22.024424 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac6da62-dc86-4d92-ade3-a8e9a19950e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:06:22.026037 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac6da62-dc86-4d92-ade3-a8e9a19950e9 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:06:22.036528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1193/2407] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:06:22.041166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f881e7d-5bbc-4861-8881-e663bc3f49ac admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:06:22.053851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f881e7d-5bbc-4861-8881-e663bc3f49ac admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:06:22.058649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb911f9-60ee-45f7-8e73-eac9c6a85812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:06:22.059370 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb911f9-60ee-45f7-8e73-eac9c6a85812 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:22.059642 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb911f9-60ee-45f7-8e73-eac9c6a85812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:22.060021 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb911f9-60ee-45f7-8e73-eac9c6a85812 admin admin] PATH_INFO: `/v3/projects/cc0ef4d1c1a746e3a2467f2d852553f8/users/19fcb43f28984cd393dc9d9e28ac7c64/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:22.063596 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb911f9-60ee-45f7-8e73-eac9c6a85812 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc0ef4d1c1a746e3a2467f2d852553f8, user_id=19fcb43f28984cd393dc9d9e28ac7c64, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:06:22.067731 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb911f9-60ee-45f7-8e73-eac9c6a85812 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:06:22.079592 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1215/2408] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:06:22 2022] PUT /identity/v3/projects/cc0ef4d1c1a746e3a2467f2d852553f8/users/19fcb43f28984cd393dc9d9e28ac7c64/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:06:22.954904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6da62-dc86-4d92-ade3-a8e9a19950e9 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:06:22.955428 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-eac6da62-dc86-4d92-ade3-a8e9a19950e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:06:23.027004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6da62-dc86-4d92-ade3-a8e9a19950e9 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:06:23.078748 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:06:23.108580 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9be0d-f572-457f-891a-ad77d0f236c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:06:23.109263 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9be0d-f572-457f-891a-ad77d0f236c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:23.109609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9be0d-f572-457f-891a-ad77d0f236c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:23.109946 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9be0d-f572-457f-891a-ad77d0f236c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:23.112167 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-16a9be0d-f572-457f-891a-ad77d0f236c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:06:23.211110 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9be0d-f572-457f-891a-ad77d0f236c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:06:23.212826 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9be0d-f572-457f-891a-ad77d0f236c2 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:06:23.225309 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1194/2409] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:06:22 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:06:28.458374 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6ca33-c379-414d-9e3a-dcd912ee2251 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:28.458974 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6ca33-c379-414d-9e3a-dcd912ee2251 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:28.458974 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6ca33-c379-414d-9e3a-dcd912ee2251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:28.489285 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-42b6ca33-c379-414d-9e3a-dcd912ee2251 None None] MFA Rules not processed for user `19fcb43f28984cd393dc9d9e28ac7c64`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:06:28.490512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-42b6ca33-c379-414d-9e3a-dcd912ee2251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:06:28.571818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:06:28.577660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1216/2410] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:06:28 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:06:28.590072 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9be0d-f572-457f-891a-ad77d0f236c2 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:06:28.591663 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-16a9be0d-f572-457f-891a-ad77d0f236c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:06:28.651270 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9be0d-f572-457f-891a-ad77d0f236c2 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:06:28.655149 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb5df1-e16c-4620-8d8e-5921c594b364 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb18199dc8b42589184042556e98983', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:06:28.655751 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb5df1-e16c-4620-8d8e-5921c594b364 service glance] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:28.656028 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb5df1-e16c-4620-8d8e-5921c594b364 service glance] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:28.656296 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb5df1-e16c-4620-8d8e-5921c594b364 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:28.737154 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb5df1-e16c-4620-8d8e-5921c594b364 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:06:28.739999 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb5df1-e16c-4620-8d8e-5921c594b364 service glance] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:06:28.749874 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1195/2411] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:06:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:06:29.032479 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d48dcd-23da-44db-8f51-5c48d1c87836 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:29.032944 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d48dcd-23da-44db-8f51-5c48d1c87836 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:29.033359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d48dcd-23da-44db-8f51-5c48d1c87836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:29.062087 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-14d48dcd-23da-44db-8f51-5c48d1c87836 None None] MFA Rules not processed for user `b0a879085a194b0687e1cdcbafd48033`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:06:29.063553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-14d48dcd-23da-44db-8f51-5c48d1c87836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:06:29.124573 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1217/2412] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Sep 11 00:06:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:06:29.278100 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb5df1-e16c-4620-8d8e-5921c594b364 service glance] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:06:29.279509 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9ccb5df1-e16c-4620-8d8e-5921c594b364 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:06:29.340100 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:06:29.364792 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb5df1-e16c-4620-8d8e-5921c594b364 service glance] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:06:29.369955 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053aeafd-0fb9-4072-84e8-52f31bccda6b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '564e65585e9543e286b6041af870cf41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:06:29.370838 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053aeafd-0fb9-4072-84e8-52f31bccda6b service swift] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:29.371390 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053aeafd-0fb9-4072-84e8-52f31bccda6b service swift] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:29.371663 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053aeafd-0fb9-4072-84e8-52f31bccda6b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:29.379690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053aeafd-0fb9-4072-84e8-52f31bccda6b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:06:29.381302 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053aeafd-0fb9-4072-84e8-52f31bccda6b service swift] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:06:29.389397 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1196/2413] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Sep 11 00:06:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 00:06:31.450810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d48dcd-23da-44db-8f51-5c48d1c87836 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:06:31.452281 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-14d48dcd-23da-44db-8f51-5c48d1c87836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:06:31.634817 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d48dcd-23da-44db-8f51-5c48d1c87836 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:06:31.650454 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d49ae-4409-4746-baa1-948322c2d7b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:06:31.652011 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d49ae-4409-4746-baa1-948322c2d7b3 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:31.652974 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d49ae-4409-4746-baa1-948322c2d7b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:31.653623 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d49ae-4409-4746-baa1-948322c2d7b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:31.656985 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f69d49ae-4409-4746-baa1-948322c2d7b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:06:31.682777 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d49ae-4409-4746-baa1-948322c2d7b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:06:31.686462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d49ae-4409-4746-baa1-948322c2d7b3 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:06:31.711270 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1218/2414] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:06:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:06:39.386150 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=12 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:06:40.937265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966eccf2-d6c4-4258-8cf1-15dc69ba2170 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:40.937886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966eccf2-d6c4-4258-8cf1-15dc69ba2170 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:40.937886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966eccf2-d6c4-4258-8cf1-15dc69ba2170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:40.960981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-966eccf2-d6c4-4258-8cf1-15dc69ba2170 None None] MFA Rules not processed for user `fd106d45a1ce4aad9c85c35d4f53af92`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:06:40.962222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-966eccf2-d6c4-4258-8cf1-15dc69ba2170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:06:40.970917 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1197/2415] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:06:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:06:41.353080 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d49ae-4409-4746-baa1-948322c2d7b3 service nova] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:06:41.410823 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d49ae-4409-4746-baa1-948322c2d7b3 service nova] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:06:41.414407 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d80db1-5028-46b1-a78b-0e99508b25ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:06:41.415091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d80db1-5028-46b1-a78b-0e99508b25ee service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:41.415368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d80db1-5028-46b1-a78b-0e99508b25ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:41.415639 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d80db1-5028-46b1-a78b-0e99508b25ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:41.417403 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-18d80db1-5028-46b1-a78b-0e99508b25ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:06:41.427052 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d80db1-5028-46b1-a78b-0e99508b25ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:06:41.428457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d80db1-5028-46b1-a78b-0e99508b25ee service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:06:41.436214 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1219/2416] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:06:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:06:41.986832 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966eccf2-d6c4-4258-8cf1-15dc69ba2170 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:06:41.988351 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-966eccf2-d6c4-4258-8cf1-15dc69ba2170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:06:42.092835 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966eccf2-d6c4-4258-8cf1-15dc69ba2170 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:06:42.097988 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6522d63f-f395-4372-b6ee-6014047808f2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cb1d85a338e40f18e61887a1becb906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:06:42.098583 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522d63f-f395-4372-b6ee-6014047808f2 service placement] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:42.098893 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522d63f-f395-4372-b6ee-6014047808f2 service placement] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:42.099162 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522d63f-f395-4372-b6ee-6014047808f2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:42.101010 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6522d63f-f395-4372-b6ee-6014047808f2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:06:42.112516 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6522d63f-f395-4372-b6ee-6014047808f2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:06:42.114433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6522d63f-f395-4372-b6ee-6014047808f2 service placement] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:06:42.125052 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1198/2417] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:06:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:06:42.838723 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d80db1-5028-46b1-a78b-0e99508b25ee service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:06:42.859550 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d80db1-5028-46b1-a78b-0e99508b25ee service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:06:42.866008 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998bea4-97e6-4fad-be9f-9ba8adaf3128 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:06:42.866809 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998bea4-97e6-4fad-be9f-9ba8adaf3128 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:42.867120 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998bea4-97e6-4fad-be9f-9ba8adaf3128 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:42.867490 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998bea4-97e6-4fad-be9f-9ba8adaf3128 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:42.869298 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-f998bea4-97e6-4fad-be9f-9ba8adaf3128 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:06:42.904070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:06:42.933591 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998bea4-97e6-4fad-be9f-9ba8adaf3128 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:06:42.935295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998bea4-97e6-4fad-be9f-9ba8adaf3128 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:06:42.945559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1220/2418] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:06:42 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:06:47.968253 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f619372-5066-49c8-a419-0f36ebeeb643 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:47.968770 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f619372-5066-49c8-a419-0f36ebeeb643 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:47.968770 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f619372-5066-49c8-a419-0f36ebeeb643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:47.990676 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-7f619372-5066-49c8-a419-0f36ebeeb643 None None] MFA Rules not processed for user `b0a879085a194b0687e1cdcbafd48033`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:06:47.991925 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7f619372-5066-49c8-a419-0f36ebeeb643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:06:47.999099 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1199/2419] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Sep 11 00:06:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:06:48.012172 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998bea4-97e6-4fad-be9f-9ba8adaf3128 service nova] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:06:48.023745 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998bea4-97e6-4fad-be9f-9ba8adaf3128 service nova] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:06:48.029018 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d2577-54ac-46c1-ad78-5335d33cffdf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '564e65585e9543e286b6041af870cf41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:06:48.029626 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d2577-54ac-46c1-ad78-5335d33cffdf service swift] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:48.029900 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d2577-54ac-46c1-ad78-5335d33cffdf service swift] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:48.030197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d2577-54ac-46c1-ad78-5335d33cffdf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:48.040030 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d2577-54ac-46c1-ad78-5335d33cffdf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:06:48.041649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d2577-54ac-46c1-ad78-5335d33cffdf service swift] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:06:48.050338 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1221/2420] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Sep 11 00:06:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 00:06:56.787488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f619372-5066-49c8-a419-0f36ebeeb643 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:06:56.787488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f619372-5066-49c8-a419-0f36ebeeb643 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:06:56.791320 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0826421-c56c-4b63-b1c9-0e9ac827f888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:06:56.797031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0826421-c56c-4b63-b1c9-0e9ac827f888 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:56.797031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0826421-c56c-4b63-b1c9-0e9ac827f888 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:56.797031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0826421-c56c-4b63-b1c9-0e9ac827f888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:56.797031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e0826421-c56c-4b63-b1c9-0e9ac827f888 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:06:56.804058 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0826421-c56c-4b63-b1c9-0e9ac827f888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:06:56.805518 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0826421-c56c-4b63-b1c9-0e9ac827f888 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:06:56.813691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1200/2421] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:06:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:06:58.046981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=20 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:06:58.370222 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d2577-54ac-46c1-ad78-5335d33cffdf service swift] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:06:58.371740 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a77d2577-54ac-46c1-ad78-5335d33cffdf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:06:58.383515 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d2577-54ac-46c1-ad78-5335d33cffdf service swift] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:06:58.387296 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5e744-416f-4675-9be4-d57d03e32b4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:06:58.388011 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5e744-416f-4675-9be4-d57d03e32b4e service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:06:58.388390 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5e744-416f-4675-9be4-d57d03e32b4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:06:58.388754 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5e744-416f-4675-9be4-d57d03e32b4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:06:58.390574 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-37b5e744-416f-4675-9be4-d57d03e32b4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:06:58.456456 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b5e744-416f-4675-9be4-d57d03e32b4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:06:58.457887 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b5e744-416f-4675-9be4-d57d03e32b4e service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:06:58.469786 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1222/2422] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:06:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:07:02.961902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0826421-c56c-4b63-b1c9-0e9ac827f888 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:02.961902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e0826421-c56c-4b63-b1c9-0e9ac827f888 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:02.977349 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0826421-c56c-4b63-b1c9-0e9ac827f888 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:02.982144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b839b-e11d-4bd7-b8b7-af12e1b2834b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:02.982848 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b839b-e11d-4bd7-b8b7-af12e1b2834b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:02.983181 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b839b-e11d-4bd7-b8b7-af12e1b2834b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:02.983927 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b839b-e11d-4bd7-b8b7-af12e1b2834b admin admin] PATH_INFO: `/v3/users/b8ceeb13c4c24ceb9ff1f27b4e45f62c` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:02.985134 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9b839b-e11d-4bd7-b8b7-af12e1b2834b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8ceeb13c4c24ceb9ff1f27b4e45f62c)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:02.987103 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9b839b-e11d-4bd7-b8b7-af12e1b2834b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:03.180693 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5e744-416f-4675-9be4-d57d03e32b4e service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:03.183098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-37b5e744-416f-4675-9be4-d57d03e32b4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:03.200834 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5e744-416f-4675-9be4-d57d03e32b4e service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:03.213322 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ba996-6a39-403f-99f1-4cea48031f94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:03.213957 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ba996-6a39-403f-99f1-4cea48031f94 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:03.214236 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ba996-6a39-403f-99f1-4cea48031f94 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:03.214525 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ba996-6a39-403f-99f1-4cea48031f94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:03.222583 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-713ba996-6a39-403f-99f1-4cea48031f94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:03.227907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3e9b839b-e11d-4bd7-b8b7-af12e1b2834b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8ceeb13c4c24ceb9ff1f27b4e45f62c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:03.240860 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ba996-6a39-403f-99f1-4cea48031f94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:03.242714 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ba996-6a39-403f-99f1-4cea48031f94 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:03.247288 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3e9b839b-e11d-4bd7-b8b7-af12e1b2834b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8ceeb13c4c24ceb9ff1f27b4e45f62c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:03.258898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1223/2423] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:07:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:07:03.266209 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3e9b839b-e11d-4bd7-b8b7-af12e1b2834b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8ceeb13c4c24ceb9ff1f27b4e45f62c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:03.342943 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3e9b839b-e11d-4bd7-b8b7-af12e1b2834b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8ceeb13c4c24ceb9ff1f27b4e45f62c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:03.343819 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3e9b839b-e11d-4bd7-b8b7-af12e1b2834b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8ceeb13c4c24ceb9ff1f27b4e45f62c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:03.352820 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1201/2424] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:07:02 2022] DELETE /identity/v3/users/b8ceeb13c4c24ceb9ff1f27b4e45f62c => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:03.681153 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ba996-6a39-403f-99f1-4cea48031f94 service nova] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:03.692333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ba996-6a39-403f-99f1-4cea48031f94 service nova] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:03.754928 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af07ee-218c-4a0a-9105-d5e87752891e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:03.755879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af07ee-218c-4a0a-9105-d5e87752891e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:03.756185 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af07ee-218c-4a0a-9105-d5e87752891e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:03.756461 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af07ee-218c-4a0a-9105-d5e87752891e admin admin] PATH_INFO: `/v3/projects/b0b35b1c14ba4a2e9b23a223628b449c` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:03.757579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af07ee-218c-4a0a-9105-d5e87752891e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0b35b1c14ba4a2e9b23a223628b449c)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:03.759494 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af07ee-218c-4a0a-9105-d5e87752891e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:03.870611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e0af07ee-218c-4a0a-9105-d5e87752891e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0b35b1c14ba4a2e9b23a223628b449c', 'request_id': 'req-e0af07ee-218c-4a0a-9105-d5e87752891e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:03.930617 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e0af07ee-218c-4a0a-9105-d5e87752891e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0b35b1c14ba4a2e9b23a223628b449c', 'request_id': 'req-e0af07ee-218c-4a0a-9105-d5e87752891e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:03.936540 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e0af07ee-218c-4a0a-9105-d5e87752891e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0b35b1c14ba4a2e9b23a223628b449c', 'request_id': 'req-e0af07ee-218c-4a0a-9105-d5e87752891e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:03.947822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e0af07ee-218c-4a0a-9105-d5e87752891e admin admin] The token cache is being invalidate because project b0b35b1c14ba4a2e9b23a223628b449c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:07:03.948103 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e0af07ee-218c-4a0a-9105-d5e87752891e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:03.948977 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e0af07ee-218c-4a0a-9105-d5e87752891e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:03.950569 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1224/2425] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:07:03 2022] DELETE /identity/v3/projects/b0b35b1c14ba4a2e9b23a223628b449c => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:03.955477 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce6b05-dd9a-4530-879a-b589874752bc None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:03.955755 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce6b05-dd9a-4530-879a-b589874752bc None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:03.956020 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce6b05-dd9a-4530-879a-b589874752bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:03.976101 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c4ce6b05-dd9a-4530-879a-b589874752bc None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:03.977697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c4ce6b05-dd9a-4530-879a-b589874752bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:04.009028 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:04.037763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1202/2426] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:04.045638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4719e-edee-4d3b-9433-e6f09b68d464 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:04.045907 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4719e-edee-4d3b-9433-e6f09b68d464 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:04.046180 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4719e-edee-4d3b-9433-e6f09b68d464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:04.067585 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-64c4719e-edee-4d3b-9433-e6f09b68d464 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:04.068798 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-64c4719e-edee-4d3b-9433-e6f09b68d464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:04.077254 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1225/2427] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:04.081482 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce6b05-dd9a-4530-879a-b589874752bc None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:04.091960 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce6b05-dd9a-4530-879a-b589874752bc None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:04.097029 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0664fb7-a476-4eef-b144-d31a2d3585b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:04.097630 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0664fb7-a476-4eef-b144-d31a2d3585b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:04.098026 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0664fb7-a476-4eef-b144-d31a2d3585b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:04.098306 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0664fb7-a476-4eef-b144-d31a2d3585b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:04.099655 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0664fb7-a476-4eef-b144-d31a2d3585b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:07:04.100385 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0664fb7-a476-4eef-b144-d31a2d3585b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:04.102985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0664fb7-a476-4eef-b144-d31a2d3585b6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:04.115118 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1203/2428] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:07:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:07:04.119769 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c4719e-edee-4d3b-9433-e6f09b68d464 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:04.125881 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:04.129286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c4719e-edee-4d3b-9433-e6f09b68d464 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:04.134432 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ef8a4-325e-4c97-94a7-16640079e368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:04.136774 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ef8a4-325e-4c97-94a7-16640079e368 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:04.136774 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ef8a4-325e-4c97-94a7-16640079e368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:04.136774 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ef8a4-325e-4c97-94a7-16640079e368 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:04.136774 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ef8a4-325e-4c97-94a7-16640079e368 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:04.139966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ef8a4-325e-4c97-94a7-16640079e368 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:04.229648 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1226/2429] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:07:04 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:07:04.233752 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0664fb7-a476-4eef-b144-d31a2d3585b6 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:04.244427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0664fb7-a476-4eef-b144-d31a2d3585b6 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:04.300388 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed3298-9c8f-439c-9162-4a62165c5493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:04.300901 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed3298-9c8f-439c-9162-4a62165c5493 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:04.301172 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed3298-9c8f-439c-9162-4a62165c5493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:04.302382 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed3298-9c8f-439c-9162-4a62165c5493 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:04.303057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed3298-9c8f-439c-9162-4a62165c5493 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:04.304965 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed3298-9c8f-439c-9162-4a62165c5493 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:04.405270 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1204/2430] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:07:04 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:07:04.409295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ef8a4-325e-4c97-94a7-16640079e368 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:04.420583 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ef8a4-325e-4c97-94a7-16640079e368 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:04.425257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df39b1f-c8bf-4a52-bd10-ae7975197166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:04.425852 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df39b1f-c8bf-4a52-bd10-ae7975197166 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:04.426121 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df39b1f-c8bf-4a52-bd10-ae7975197166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:04.426390 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df39b1f-c8bf-4a52-bd10-ae7975197166 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:04.427347 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df39b1f-c8bf-4a52-bd10-ae7975197166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:04.429102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df39b1f-c8bf-4a52-bd10-ae7975197166 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:04.439487 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1227/2431] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:07:04.443725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed3298-9c8f-439c-9162-4a62165c5493 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:04.454314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed3298-9c8f-439c-9162-4a62165c5493 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:04.459923 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca277f-dd70-4432-8368-fd0c87860da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:04.460657 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca277f-dd70-4432-8368-fd0c87860da8 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:04.460936 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca277f-dd70-4432-8368-fd0c87860da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:04.461209 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca277f-dd70-4432-8368-fd0c87860da8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:04.461852 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca277f-dd70-4432-8368-fd0c87860da8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:04.463522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca277f-dd70-4432-8368-fd0c87860da8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:04.475339 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1205/2432] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:07:04.479516 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df39b1f-c8bf-4a52-bd10-ae7975197166 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:04.489190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df39b1f-c8bf-4a52-bd10-ae7975197166 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:04.494617 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacb287-c934-47ce-8fe7-1bc46174bed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:04.495592 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacb287-c934-47ce-8fe7-1bc46174bed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:04.495592 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacb287-c934-47ce-8fe7-1bc46174bed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:04.495846 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacb287-c934-47ce-8fe7-1bc46174bed7 admin admin] PATH_INFO: `/v3/projects/6e41ad56c0424e968f513b1175c599d9/users/97b33ea3125442d187b9d8d4b169aab5/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:04.514644 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdacb287-c934-47ce-8fe7-1bc46174bed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e41ad56c0424e968f513b1175c599d9, user_id=97b33ea3125442d187b9d8d4b169aab5, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:04.517298 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdacb287-c934-47ce-8fe7-1bc46174bed7 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:04.577158 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1228/2433] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:07:04 2022] PUT /identity/v3/projects/6e41ad56c0424e968f513b1175c599d9/users/97b33ea3125442d187b9d8d4b169aab5/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:04.590491 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2576ee6-761c-46d9-a255-b6eeb37ddcfe None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:04.590803 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2576ee6-761c-46d9-a255-b6eeb37ddcfe None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:04.591079 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2576ee6-761c-46d9-a255-b6eeb37ddcfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:04.621705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f2576ee6-761c-46d9-a255-b6eeb37ddcfe None None] MFA Rules not processed for user `97b33ea3125442d187b9d8d4b169aab5`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:04.622949 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f2576ee6-761c-46d9-a255-b6eeb37ddcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:04.690612 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1206/2434] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:04.702963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacb287-c934-47ce-8fe7-1bc46174bed7 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:04.704623 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cdacb287-c934-47ce-8fe7-1bc46174bed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:04.803269 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacb287-c934-47ce-8fe7-1bc46174bed7 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:04.809765 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de7734-9251-4163-a814-13b9cb065ddd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:04.810466 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de7734-9251-4163-a814-13b9cb065ddd service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:04.810847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de7734-9251-4163-a814-13b9cb065ddd service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:04.811210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de7734-9251-4163-a814-13b9cb065ddd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:04.820431 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de7734-9251-4163-a814-13b9cb065ddd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:04.822109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de7734-9251-4163-a814-13b9cb065ddd service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:04.834206 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1229/2435] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:07:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:07:05.070798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2576ee6-761c-46d9-a255-b6eeb37ddcfe None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:05.104205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:05.132828 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2576ee6-761c-46d9-a255-b6eeb37ddcfe None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:05.138173 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f511e-2417-4d70-b590-8fdaa32c05ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:05.138841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f511e-2417-4d70-b590-8fdaa32c05ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:05.139185 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f511e-2417-4d70-b590-8fdaa32c05ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:05.139361 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f511e-2417-4d70-b590-8fdaa32c05ed admin admin] PATH_INFO: `/v3/users/97b33ea3125442d187b9d8d4b169aab5` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:05.140471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f511e-2417-4d70-b590-8fdaa32c05ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=97b33ea3125442d187b9d8d4b169aab5)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:05.142388 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f511e-2417-4d70-b590-8fdaa32c05ed admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:05.306291 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4c0f511e-2417-4d70-b590-8fdaa32c05ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97b33ea3125442d187b9d8d4b169aab5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:05.313512 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4c0f511e-2417-4d70-b590-8fdaa32c05ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97b33ea3125442d187b9d8d4b169aab5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:05.326008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4c0f511e-2417-4d70-b590-8fdaa32c05ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97b33ea3125442d187b9d8d4b169aab5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:05.375760 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4c0f511e-2417-4d70-b590-8fdaa32c05ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97b33ea3125442d187b9d8d4b169aab5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:05.376294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4c0f511e-2417-4d70-b590-8fdaa32c05ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97b33ea3125442d187b9d8d4b169aab5'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:05.383521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1207/2436] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:07:05 2022] DELETE /identity/v3/users/97b33ea3125442d187b9d8d4b169aab5 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:05.388025 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006686b-0242-4d58-bcbc-6b0f04a54bf3 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:05.388529 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006686b-0242-4d58-bcbc-6b0f04a54bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:05.388626 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006686b-0242-4d58-bcbc-6b0f04a54bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:05.407815 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-8006686b-0242-4d58-bcbc-6b0f04a54bf3 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:05.409000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8006686b-0242-4d58-bcbc-6b0f04a54bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:05.468883 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1230/2437] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:05.478826 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f511e-2417-4d70-b590-8fdaa32c05ed admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:05.482101 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4c0f511e-2417-4d70-b590-8fdaa32c05ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:05.539021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f511e-2417-4d70-b590-8fdaa32c05ed admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:05.543057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5ecfa-c9a3-40fa-bb61-620eaead43ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:05.543487 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5ecfa-c9a3-40fa-bb61-620eaead43ea service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:05.543637 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5ecfa-c9a3-40fa-bb61-620eaead43ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:05.543910 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5ecfa-c9a3-40fa-bb61-620eaead43ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:05.557117 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5ecfa-c9a3-40fa-bb61-620eaead43ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:05.557117 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5ecfa-c9a3-40fa-bb61-620eaead43ea service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:05.566839 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1208/2438] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:07:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:07:06.297222 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006686b-0242-4d58-bcbc-6b0f04a54bf3 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:06.298634 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8006686b-0242-4d58-bcbc-6b0f04a54bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:06.313265 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006686b-0242-4d58-bcbc-6b0f04a54bf3 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:06.322626 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36887fd8-63ca-41e2-a2d2-a2bc15e01a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:06.323309 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36887fd8-63ca-41e2-a2d2-a2bc15e01a75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:06.324086 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36887fd8-63ca-41e2-a2d2-a2bc15e01a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:06.324383 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36887fd8-63ca-41e2-a2d2-a2bc15e01a75 admin admin] PATH_INFO: `/v3/projects/6e41ad56c0424e968f513b1175c599d9` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:06.325805 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36887fd8-63ca-41e2-a2d2-a2bc15e01a75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e41ad56c0424e968f513b1175c599d9)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:06.327507 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36887fd8-63ca-41e2-a2d2-a2bc15e01a75 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:06.454116 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:06.477959 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-36887fd8-63ca-41e2-a2d2-a2bc15e01a75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e41ad56c0424e968f513b1175c599d9', 'request_id': 'req-36887fd8-63ca-41e2-a2d2-a2bc15e01a75'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:06.565302 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-36887fd8-63ca-41e2-a2d2-a2bc15e01a75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e41ad56c0424e968f513b1175c599d9', 'request_id': 'req-36887fd8-63ca-41e2-a2d2-a2bc15e01a75'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:06.569172 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539439b-c5a5-4b03-9c9d-bde8c13159ba None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:06.572057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539439b-c5a5-4b03-9c9d-bde8c13159ba None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:06.576046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-36887fd8-63ca-41e2-a2d2-a2bc15e01a75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e41ad56c0424e968f513b1175c599d9', 'request_id': 'req-36887fd8-63ca-41e2-a2d2-a2bc15e01a75'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:06.576184 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539439b-c5a5-4b03-9c9d-bde8c13159ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:06.588362 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-36887fd8-63ca-41e2-a2d2-a2bc15e01a75 admin admin] The token cache is being invalidate because project 6e41ad56c0424e968f513b1175c599d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:07:06.588709 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-36887fd8-63ca-41e2-a2d2-a2bc15e01a75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:06.589805 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-36887fd8-63ca-41e2-a2d2-a2bc15e01a75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:06.595101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1231/2439] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:07:06 2022] DELETE /identity/v3/projects/6e41ad56c0424e968f513b1175c599d9 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:06.596454 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de563c-012f-455e-bd6b-f4624e8a4a3b None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:06.596737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de563c-012f-455e-bd6b-f4624e8a4a3b None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:06.597010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de563c-012f-455e-bd6b-f4624e8a4a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:06.601371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-4539439b-c5a5-4b03-9c9d-bde8c13159ba None None] MFA Rules not processed for user `b0a879085a194b0687e1cdcbafd48033`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:06.603418 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4539439b-c5a5-4b03-9c9d-bde8c13159ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:06.630027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-51de563c-012f-455e-bd6b-f4624e8a4a3b None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:06.631386 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-51de563c-012f-455e-bd6b-f4624e8a4a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:06.672134 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:06.682461 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1209/2440] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Sep 11 00:07:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:06.697522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539439b-c5a5-4b03-9c9d-bde8c13159ba None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:06.700535 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4539439b-c5a5-4b03-9c9d-bde8c13159ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:06.720342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1232/2441] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:06.728426 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895f183-f665-401e-95c6-41d9289eb43a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:06.728720 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895f183-f665-401e-95c6-41d9289eb43a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:06.729027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895f183-f665-401e-95c6-41d9289eb43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:06.791789 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-5895f183-f665-401e-95c6-41d9289eb43a None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:06.793367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5895f183-f665-401e-95c6-41d9289eb43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:06.803077 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1233/2442] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:06.807711 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5895f183-f665-401e-95c6-41d9289eb43a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:06.819964 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5895f183-f665-401e-95c6-41d9289eb43a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:06.825597 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271bab1-6bd6-4938-a89b-2c1ef3eba674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:06.826287 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271bab1-6bd6-4938-a89b-2c1ef3eba674 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:06.826475 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271bab1-6bd6-4938-a89b-2c1ef3eba674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:06.827073 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271bab1-6bd6-4938-a89b-2c1ef3eba674 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:06.827736 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271bab1-6bd6-4938-a89b-2c1ef3eba674 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:07:06.828039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271bab1-6bd6-4938-a89b-2c1ef3eba674 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:06.829629 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271bab1-6bd6-4938-a89b-2c1ef3eba674 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:06.871594 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1234/2443] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:07:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:07:06.876300 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271bab1-6bd6-4938-a89b-2c1ef3eba674 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:06.881955 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539439b-c5a5-4b03-9c9d-bde8c13159ba None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:06.887998 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271bab1-6bd6-4938-a89b-2c1ef3eba674 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:06.893506 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9136420-edd1-424d-9242-dda56f223c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:06.894841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc9ae1-7ab2-4277-8631-a6409dcbcce9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '564e65585e9543e286b6041af870cf41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:06.895626 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc9ae1-7ab2-4277-8631-a6409dcbcce9 service swift] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:06.896062 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc9ae1-7ab2-4277-8631-a6409dcbcce9 service swift] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:06.896294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc9ae1-7ab2-4277-8631-a6409dcbcce9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:06.896577 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9136420-edd1-424d-9242-dda56f223c32 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:06.896923 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9136420-edd1-424d-9242-dda56f223c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:06.897200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9136420-edd1-424d-9242-dda56f223c32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:06.897864 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9136420-edd1-424d-9242-dda56f223c32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:06.899820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9136420-edd1-424d-9242-dda56f223c32 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:06.915833 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc9ae1-7ab2-4277-8631-a6409dcbcce9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:06.917442 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc9ae1-7ab2-4277-8631-a6409dcbcce9 service swift] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:06.931081 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1210/2444] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Sep 11 00:07:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 00:07:06.992825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1235/2445] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:07:06 2022] POST /identity/v3/projects => generated 405 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:07:06.997592 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc9ae1-7ab2-4277-8631-a6409dcbcce9 service swift] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:07.017213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc9ae1-7ab2-4277-8631-a6409dcbcce9 service swift] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:07.108116 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe8300-4100-4957-bf78-aced2a063155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:07.109678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe8300-4100-4957-bf78-aced2a063155 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:07.110125 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe8300-4100-4957-bf78-aced2a063155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:07.110577 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe8300-4100-4957-bf78-aced2a063155 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:07.112188 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe8300-4100-4957-bf78-aced2a063155 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:07.116804 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe8300-4100-4957-bf78-aced2a063155 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:07.237162 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1211/2446] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:07:06 2022] POST /identity/v3/users => generated 303 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:07:07.240137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9136420-edd1-424d-9242-dda56f223c32 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:07.251738 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9136420-edd1-424d-9242-dda56f223c32 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:07.261650 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444e6f4-523f-4111-a4ab-88f0ff12904f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:07.265424 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444e6f4-523f-4111-a4ab-88f0ff12904f admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:07.265424 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444e6f4-523f-4111-a4ab-88f0ff12904f admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:07.265424 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444e6f4-523f-4111-a4ab-88f0ff12904f admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:07.265424 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b444e6f4-523f-4111-a4ab-88f0ff12904f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:07.265424 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b444e6f4-523f-4111-a4ab-88f0ff12904f admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:07.276928 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1236/2447] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:07:07.281608 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe8300-4100-4957-bf78-aced2a063155 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:07.295698 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe8300-4100-4957-bf78-aced2a063155 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:07.305972 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0a8c3-3dd8-4de1-9f62-d640c27dc5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:07.306774 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0a8c3-3dd8-4de1-9f62-d640c27dc5c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:07.307155 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0a8c3-3dd8-4de1-9f62-d640c27dc5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:07.307483 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0a8c3-3dd8-4de1-9f62-d640c27dc5c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:07.308200 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0a8c3-3dd8-4de1-9f62-d640c27dc5c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:07.310296 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0a8c3-3dd8-4de1-9f62-d640c27dc5c1 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:07.333985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1212/2448] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:07:07.338828 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444e6f4-523f-4111-a4ab-88f0ff12904f admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:07.351055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444e6f4-523f-4111-a4ab-88f0ff12904f admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:07.357180 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8880f2a-0997-4ff1-be46-779293be6455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:07.357853 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8880f2a-0997-4ff1-be46-779293be6455 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:07.358152 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8880f2a-0997-4ff1-be46-779293be6455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:07.358644 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8880f2a-0997-4ff1-be46-779293be6455 admin admin] PATH_INFO: `/v3/projects/fa662000f0f84613aa8a8c4cc12748b4/users/f05949ad11ff4fd59081583c88c59a50/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:07.377659 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8880f2a-0997-4ff1-be46-779293be6455 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa662000f0f84613aa8a8c4cc12748b4, user_id=f05949ad11ff4fd59081583c88c59a50, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:07.381010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8880f2a-0997-4ff1-be46-779293be6455 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:07.581322 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1237/2449] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:07:07 2022] PUT /identity/v3/projects/fa662000f0f84613aa8a8c4cc12748b4/users/f05949ad11ff4fd59081583c88c59a50/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:07:07.586096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cdea8-2e05-49c2-9c61-d1300d6cf31a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:07.586698 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cdea8-2e05-49c2-9c61-d1300d6cf31a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:07.586698 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cdea8-2e05-49c2-9c61-d1300d6cf31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:07.610749 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f22cdea8-2e05-49c2-9c61-d1300d6cf31a None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:07.611991 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f22cdea8-2e05-49c2-9c61-d1300d6cf31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:07.681611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1213/2450] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:07.692008 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8880f2a-0997-4ff1-be46-779293be6455 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:07.694356 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a8880f2a-0997-4ff1-be46-779293be6455 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:07.744896 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:07.760887 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8880f2a-0997-4ff1-be46-779293be6455 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:07.771944 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eff1d2-ba6e-4a55-9c2d-4e66b54c31db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:07.772575 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eff1d2-ba6e-4a55-9c2d-4e66b54c31db service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:07.772849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eff1d2-ba6e-4a55-9c2d-4e66b54c31db service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:07.773116 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eff1d2-ba6e-4a55-9c2d-4e66b54c31db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:07.783666 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eff1d2-ba6e-4a55-9c2d-4e66b54c31db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:07.785191 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eff1d2-ba6e-4a55-9c2d-4e66b54c31db service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:07.807638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1238/2451] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:07:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:07:09.259891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9531f-5593-473b-b161-e7fcb952a565 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:09.260488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9531f-5593-473b-b161-e7fcb952a565 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:09.261688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9531f-5593-473b-b161-e7fcb952a565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:09.289405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:09.311561 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-3bf9531f-5593-473b-b161-e7fcb952a565 None None] MFA Rules not processed for user `f05949ad11ff4fd59081583c88c59a50`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:09.313455 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3bf9531f-5593-473b-b161-e7fcb952a565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:09.406610 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1214/2452] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:09 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:09.410497 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eff1d2-ba6e-4a55-9c2d-4e66b54c31db service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:09.420545 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eff1d2-ba6e-4a55-9c2d-4e66b54c31db service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:09.442776 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fa4e4-61df-4bff-84a9-52931d084629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:09.443409 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fa4e4-61df-4bff-84a9-52931d084629 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:09.443679 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fa4e4-61df-4bff-84a9-52931d084629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:09.444152 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fa4e4-61df-4bff-84a9-52931d084629 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:09.444801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81fa4e4-61df-4bff-84a9-52931d084629 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:09.447108 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81fa4e4-61df-4bff-84a9-52931d084629 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:09.559262 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1239/2453] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:07:09 2022] POST /identity/v3/projects => generated 405 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:07:09.562978 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9531f-5593-473b-b161-e7fcb952a565 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:09.577315 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9531f-5593-473b-b161-e7fcb952a565 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:09.666193 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70782044-35c7-46bc-8cbf-5d86300b6ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:09.667623 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70782044-35c7-46bc-8cbf-5d86300b6ee4 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:09.668129 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70782044-35c7-46bc-8cbf-5d86300b6ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:09.668404 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70782044-35c7-46bc-8cbf-5d86300b6ee4 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:09.669041 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70782044-35c7-46bc-8cbf-5d86300b6ee4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:09.671064 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70782044-35c7-46bc-8cbf-5d86300b6ee4 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:09.776536 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1215/2454] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:07:09 2022] POST /identity/v3/users => generated 309 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:07:09.777710 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fa4e4-61df-4bff-84a9-52931d084629 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:09.791450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fa4e4-61df-4bff-84a9-52931d084629 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:09.799221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be53f2-2779-4b6e-a468-b2d6196294e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:09.799221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be53f2-2779-4b6e-a468-b2d6196294e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:09.799221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be53f2-2779-4b6e-a468-b2d6196294e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:09.799221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be53f2-2779-4b6e-a468-b2d6196294e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:09.799643 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be53f2-2779-4b6e-a468-b2d6196294e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:09.800883 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be53f2-2779-4b6e-a468-b2d6196294e8 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:09.815393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1240/2455] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:07:09.820335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70782044-35c7-46bc-8cbf-5d86300b6ee4 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:09.835507 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70782044-35c7-46bc-8cbf-5d86300b6ee4 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:09.841655 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0674b1-b967-414b-a983-ea8167d8c57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:09.842328 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0674b1-b967-414b-a983-ea8167d8c57a admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:09.842608 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0674b1-b967-414b-a983-ea8167d8c57a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:09.842910 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0674b1-b967-414b-a983-ea8167d8c57a admin admin] PATH_INFO: `/v3/projects/ad05dfb0b0e94bb0a2ddaf7eed60b860/users/a5419366ea904abb861d6365c13baf31/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:09.863673 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0674b1-b967-414b-a983-ea8167d8c57a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad05dfb0b0e94bb0a2ddaf7eed60b860, user_id=a5419366ea904abb861d6365c13baf31, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:09.866472 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0674b1-b967-414b-a983-ea8167d8c57a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:09.949778 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1216/2456] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:07:09 2022] PUT /identity/v3/projects/ad05dfb0b0e94bb0a2ddaf7eed60b860/users/a5419366ea904abb861d6365c13baf31/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:09.954931 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be53f2-2779-4b6e-a468-b2d6196294e8 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:09.968229 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be53f2-2779-4b6e-a468-b2d6196294e8 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:10.035592 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc154276-8ddc-43da-95bd-c35f52aafabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:10.037424 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc154276-8ddc-43da-95bd-c35f52aafabc admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:10.037903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc154276-8ddc-43da-95bd-c35f52aafabc admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:10.038180 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc154276-8ddc-43da-95bd-c35f52aafabc admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:10.039031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc154276-8ddc-43da-95bd-c35f52aafabc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:10.040543 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc154276-8ddc-43da-95bd-c35f52aafabc admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:10.068319 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1241/2457] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:07:10.072360 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0674b1-b967-414b-a983-ea8167d8c57a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:10.083842 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0674b1-b967-414b-a983-ea8167d8c57a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:10.089080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb8672-60d6-4a3d-8ae9-aa4a33e61c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:10.089752 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb8672-60d6-4a3d-8ae9-aa4a33e61c95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:10.090029 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb8672-60d6-4a3d-8ae9-aa4a33e61c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:10.090306 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb8672-60d6-4a3d-8ae9-aa4a33e61c95 admin admin] PATH_INFO: `/v3/projects/ad05dfb0b0e94bb0a2ddaf7eed60b860/users/a5419366ea904abb861d6365c13baf31/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:10.092299 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb8672-60d6-4a3d-8ae9-aa4a33e61c95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad05dfb0b0e94bb0a2ddaf7eed60b860, user_id=a5419366ea904abb861d6365c13baf31, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:10.095125 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb8672-60d6-4a3d-8ae9-aa4a33e61c95 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:10.109518 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1217/2458] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:07:10 2022] PUT /identity/v3/projects/ad05dfb0b0e94bb0a2ddaf7eed60b860/users/a5419366ea904abb861d6365c13baf31/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:11.838149 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db90ed-3958-4531-8ea8-d8aa1b4545d8 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:11.838706 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db90ed-3958-4531-8ea8-d8aa1b4545d8 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:11.838931 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db90ed-3958-4531-8ea8-d8aa1b4545d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:11.889938 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-92db90ed-3958-4531-8ea8-d8aa1b4545d8 None None] MFA Rules not processed for user `a5419366ea904abb861d6365c13baf31`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:11.891271 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-92db90ed-3958-4531-8ea8-d8aa1b4545d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:12.035409 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1242/2459] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:11 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:13.687663 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb8672-60d6-4a3d-8ae9-aa4a33e61c95 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:13.746109 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:13.760816 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb8672-60d6-4a3d-8ae9-aa4a33e61c95 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:13.765892 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4925f2-a9b5-4228-89b0-336d13117d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:13.766563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4925f2-a9b5-4228-89b0-336d13117d3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:13.766877 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4925f2-a9b5-4228-89b0-336d13117d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:13.767165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4925f2-a9b5-4228-89b0-336d13117d3a admin admin] PATH_INFO: `/v3/users/f05949ad11ff4fd59081583c88c59a50` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:13.768283 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4925f2-a9b5-4228-89b0-336d13117d3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f05949ad11ff4fd59081583c88c59a50)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:13.770249 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4925f2-a9b5-4228-89b0-336d13117d3a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:14.222880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7c4925f2-a9b5-4228-89b0-336d13117d3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f05949ad11ff4fd59081583c88c59a50'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:14.239094 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7c4925f2-a9b5-4228-89b0-336d13117d3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f05949ad11ff4fd59081583c88c59a50'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:14.253115 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7c4925f2-a9b5-4228-89b0-336d13117d3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f05949ad11ff4fd59081583c88c59a50'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:14.320091 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7c4925f2-a9b5-4228-89b0-336d13117d3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f05949ad11ff4fd59081583c88c59a50'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:14.320728 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7c4925f2-a9b5-4228-89b0-336d13117d3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f05949ad11ff4fd59081583c88c59a50'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:14.328657 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1218/2460] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:07:13 2022] DELETE /identity/v3/users/f05949ad11ff4fd59081583c88c59a50 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:14.623238 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db90ed-3958-4531-8ea8-d8aa1b4545d8 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:14.642920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db90ed-3958-4531-8ea8-d8aa1b4545d8 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:14.710110 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:14.756408 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee73171-0855-44c2-9655-4c8ef2533a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:14.760741 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee73171-0855-44c2-9655-4c8ef2533a47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:14.760741 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee73171-0855-44c2-9655-4c8ef2533a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:14.760741 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee73171-0855-44c2-9655-4c8ef2533a47 admin admin] PATH_INFO: `/v3/projects/fa662000f0f84613aa8a8c4cc12748b4` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:14.760741 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee73171-0855-44c2-9655-4c8ef2533a47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa662000f0f84613aa8a8c4cc12748b4)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:14.762380 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee73171-0855-44c2-9655-4c8ef2533a47 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:14.913928 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-bee73171-0855-44c2-9655-4c8ef2533a47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa662000f0f84613aa8a8c4cc12748b4', 'request_id': 'req-bee73171-0855-44c2-9655-4c8ef2533a47'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:14.982275 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-bee73171-0855-44c2-9655-4c8ef2533a47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa662000f0f84613aa8a8c4cc12748b4', 'request_id': 'req-bee73171-0855-44c2-9655-4c8ef2533a47'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:14.993076 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-bee73171-0855-44c2-9655-4c8ef2533a47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa662000f0f84613aa8a8c4cc12748b4', 'request_id': 'req-bee73171-0855-44c2-9655-4c8ef2533a47'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:15.020373 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-bee73171-0855-44c2-9655-4c8ef2533a47 admin admin] The token cache is being invalidate because project fa662000f0f84613aa8a8c4cc12748b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:07:15.020695 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-bee73171-0855-44c2-9655-4c8ef2533a47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:15.023552 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-bee73171-0855-44c2-9655-4c8ef2533a47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:15.029841 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1243/2461] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:07:14 2022] DELETE /identity/v3/projects/fa662000f0f84613aa8a8c4cc12748b4 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:15.034802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4925f2-a9b5-4228-89b0-336d13117d3a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:15.038794 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7c4925f2-a9b5-4228-89b0-336d13117d3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:15.078313 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4925f2-a9b5-4228-89b0-336d13117d3a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:15.178684 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c207a-02e3-475e-b50c-12b9f96224c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:15.179356 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c207a-02e3-475e-b50c-12b9f96224c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:15.179631 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c207a-02e3-475e-b50c-12b9f96224c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:15.179896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c207a-02e3-475e-b50c-12b9f96224c4 admin admin] PATH_INFO: `/v3/users/a5419366ea904abb861d6365c13baf31` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:15.181050 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c207a-02e3-475e-b50c-12b9f96224c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5419366ea904abb861d6365c13baf31)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:15.183037 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c207a-02e3-475e-b50c-12b9f96224c4 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:15.350209 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c27c207a-02e3-475e-b50c-12b9f96224c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5419366ea904abb861d6365c13baf31'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:15.361144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c27c207a-02e3-475e-b50c-12b9f96224c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5419366ea904abb861d6365c13baf31'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:15.372807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:15.375246 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c27c207a-02e3-475e-b50c-12b9f96224c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5419366ea904abb861d6365c13baf31'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:15.434341 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c27c207a-02e3-475e-b50c-12b9f96224c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5419366ea904abb861d6365c13baf31'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:15.435185 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c27c207a-02e3-475e-b50c-12b9f96224c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5419366ea904abb861d6365c13baf31'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:15.441869 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1219/2462] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:07:15 2022] DELETE /identity/v3/users/a5419366ea904abb861d6365c13baf31 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:15.772252 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee73171-0855-44c2-9655-4c8ef2533a47 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:15.775940 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-bee73171-0855-44c2-9655-4c8ef2533a47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:15.791501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee73171-0855-44c2-9655-4c8ef2533a47 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:15.879036 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aefafa1-c7ad-4c82-8706-0170d34c7d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:15.880053 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefafa1-c7ad-4c82-8706-0170d34c7d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:15.881223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefafa1-c7ad-4c82-8706-0170d34c7d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:15.881499 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefafa1-c7ad-4c82-8706-0170d34c7d56 admin admin] PATH_INFO: `/v3/projects/ad05dfb0b0e94bb0a2ddaf7eed60b860` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:15.883482 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aefafa1-c7ad-4c82-8706-0170d34c7d56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad05dfb0b0e94bb0a2ddaf7eed60b860)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:15.885715 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aefafa1-c7ad-4c82-8706-0170d34c7d56 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:16.050931 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4aefafa1-c7ad-4c82-8706-0170d34c7d56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad05dfb0b0e94bb0a2ddaf7eed60b860', 'request_id': 'req-4aefafa1-c7ad-4c82-8706-0170d34c7d56'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:16.152808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4aefafa1-c7ad-4c82-8706-0170d34c7d56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad05dfb0b0e94bb0a2ddaf7eed60b860', 'request_id': 'req-4aefafa1-c7ad-4c82-8706-0170d34c7d56'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:16.158932 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:16.162918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4aefafa1-c7ad-4c82-8706-0170d34c7d56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad05dfb0b0e94bb0a2ddaf7eed60b860', 'request_id': 'req-4aefafa1-c7ad-4c82-8706-0170d34c7d56'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:16.175890 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4aefafa1-c7ad-4c82-8706-0170d34c7d56 admin admin] The token cache is being invalidate because project ad05dfb0b0e94bb0a2ddaf7eed60b860 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:07:16.176265 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4aefafa1-c7ad-4c82-8706-0170d34c7d56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:16.176992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4aefafa1-c7ad-4c82-8706-0170d34c7d56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:16.181931 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1244/2463] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:07:15 2022] DELETE /identity/v3/projects/ad05dfb0b0e94bb0a2ddaf7eed60b860 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:16.187981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126016d-8a55-48cd-8bd2-521d39dd59c9 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:16.188461 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126016d-8a55-48cd-8bd2-521d39dd59c9 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:16.188738 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126016d-8a55-48cd-8bd2-521d39dd59c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:16.215473 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f126016d-8a55-48cd-8bd2-521d39dd59c9 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:16.219434 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f126016d-8a55-48cd-8bd2-521d39dd59c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:16.290303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1220/2464] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:16.298543 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ec30d-40c5-4544-a804-725311121527 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:16.299001 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ec30d-40c5-4544-a804-725311121527 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:16.299001 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ec30d-40c5-4544-a804-725311121527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:16.329176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-6f4ec30d-40c5-4544-a804-725311121527 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:16.330563 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6f4ec30d-40c5-4544-a804-725311121527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:16.342010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1245/2465] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:16.346212 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126016d-8a55-48cd-8bd2-521d39dd59c9 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:16.361162 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126016d-8a55-48cd-8bd2-521d39dd59c9 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:16.366151 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44ca66-02ac-487b-b3ad-e0cd3e8e929a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:16.366801 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44ca66-02ac-487b-b3ad-e0cd3e8e929a admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:16.367084 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44ca66-02ac-487b-b3ad-e0cd3e8e929a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:16.367356 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44ca66-02ac-487b-b3ad-e0cd3e8e929a admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:16.368007 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c44ca66-02ac-487b-b3ad-e0cd3e8e929a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:07:16.368298 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c44ca66-02ac-487b-b3ad-e0cd3e8e929a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:16.369930 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c44ca66-02ac-487b-b3ad-e0cd3e8e929a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:16.383038 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1221/2466] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:07:16.388503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ec30d-40c5-4544-a804-725311121527 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:16.400254 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ec30d-40c5-4544-a804-725311121527 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:16.410097 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3769ff2-38db-4660-9927-09e601a27e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:16.410710 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3769ff2-38db-4660-9927-09e601a27e94 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:16.410987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3769ff2-38db-4660-9927-09e601a27e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:16.411252 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3769ff2-38db-4660-9927-09e601a27e94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:16.411974 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3769ff2-38db-4660-9927-09e601a27e94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:16.414282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3769ff2-38db-4660-9927-09e601a27e94 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:16.523759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1246/2467] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:07:16 2022] POST /identity/v3/projects => generated 363 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:07:16.530905 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44ca66-02ac-487b-b3ad-e0cd3e8e929a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:16.542855 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44ca66-02ac-487b-b3ad-e0cd3e8e929a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:16.612122 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab41713-8a45-469f-9e65-efcaaa6974ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:16.613160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab41713-8a45-469f-9e65-efcaaa6974ad admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:16.613442 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab41713-8a45-469f-9e65-efcaaa6974ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:16.613715 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab41713-8a45-469f-9e65-efcaaa6974ad admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:16.614401 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab41713-8a45-469f-9e65-efcaaa6974ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:16.617192 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab41713-8a45-469f-9e65-efcaaa6974ad admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:16.726620 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1222/2468] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:07:16 2022] POST /identity/v3/users => generated 282 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:07:16.733254 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3769ff2-38db-4660-9927-09e601a27e94 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:16.744650 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3769ff2-38db-4660-9927-09e601a27e94 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:16.751728 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5172d92-89ad-4bcc-b1a9-a75cfcf68257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:16.752439 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5172d92-89ad-4bcc-b1a9-a75cfcf68257 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:16.752697 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5172d92-89ad-4bcc-b1a9-a75cfcf68257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:16.753067 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5172d92-89ad-4bcc-b1a9-a75cfcf68257 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:16.753779 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5172d92-89ad-4bcc-b1a9-a75cfcf68257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:16.756095 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5172d92-89ad-4bcc-b1a9-a75cfcf68257 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:16.768907 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1247/2469] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:07:16.775060 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab41713-8a45-469f-9e65-efcaaa6974ad admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:16.785933 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab41713-8a45-469f-9e65-efcaaa6974ad admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:16.793162 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9626b831-b1bc-432d-ba00-f27c1b9e2bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:16.794096 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626b831-b1bc-432d-ba00-f27c1b9e2bf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:16.794369 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626b831-b1bc-432d-ba00-f27c1b9e2bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:16.794640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626b831-b1bc-432d-ba00-f27c1b9e2bf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:16.795321 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9626b831-b1bc-432d-ba00-f27c1b9e2bf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:16.796920 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9626b831-b1bc-432d-ba00-f27c1b9e2bf3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:16.810709 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1223/2470] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:07:16.814378 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5172d92-89ad-4bcc-b1a9-a75cfcf68257 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:16.826112 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5172d92-89ad-4bcc-b1a9-a75cfcf68257 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:16.830824 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3bbca-4909-46af-92a6-7dfc4113a8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:16.831483 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3bbca-4909-46af-92a6-7dfc4113a8b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:16.831895 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3bbca-4909-46af-92a6-7dfc4113a8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:16.832178 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3bbca-4909-46af-92a6-7dfc4113a8b5 admin admin] PATH_INFO: `/v3/projects/4e181a30cb154859a7a283dffb8d4c2c/users/33f37811ea854ebf8f77e54e258f7691/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:16.854428 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3bbca-4909-46af-92a6-7dfc4113a8b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e181a30cb154859a7a283dffb8d4c2c, user_id=33f37811ea854ebf8f77e54e258f7691, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:16.857195 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3bbca-4909-46af-92a6-7dfc4113a8b5 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:16.927408 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1248/2471] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:07:16 2022] PUT /identity/v3/projects/4e181a30cb154859a7a283dffb8d4c2c/users/33f37811ea854ebf8f77e54e258f7691/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:16.945365 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456b143-b2d0-45f5-82f1-fcadb0ef5245 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:16.945679 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456b143-b2d0-45f5-82f1-fcadb0ef5245 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:16.946265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456b143-b2d0-45f5-82f1-fcadb0ef5245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:16.980886 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-0456b143-b2d0-45f5-82f1-fcadb0ef5245 None None] MFA Rules not processed for user `33f37811ea854ebf8f77e54e258f7691`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:16.982158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0456b143-b2d0-45f5-82f1-fcadb0ef5245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:17.032471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:17.070131 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1224/2472] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:17.075154 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3bbca-4909-46af-92a6-7dfc4113a8b5 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:17.154803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3bbca-4909-46af-92a6-7dfc4113a8b5 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:17.264363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe9646-4fbf-4f81-a59f-55bd78c2ed8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:17.264363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe9646-4fbf-4f81-a59f-55bd78c2ed8c admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:17.264363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe9646-4fbf-4f81-a59f-55bd78c2ed8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:17.264363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe9646-4fbf-4f81-a59f-55bd78c2ed8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:17.264973 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe9646-4fbf-4f81-a59f-55bd78c2ed8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:17.264973 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe9646-4fbf-4f81-a59f-55bd78c2ed8c admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:17.475691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1249/2473] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:07:17 2022] POST /identity/v3/projects => generated 365 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:07:17.479785 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456b143-b2d0-45f5-82f1-fcadb0ef5245 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:17.490826 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456b143-b2d0-45f5-82f1-fcadb0ef5245 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:17.556972 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725c73b-13e6-4b14-8bde-683b4f74a16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:17.557211 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725c73b-13e6-4b14-8bde-683b4f74a16a admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:17.557314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725c73b-13e6-4b14-8bde-683b4f74a16a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:17.557581 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725c73b-13e6-4b14-8bde-683b4f74a16a admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:17.558217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e725c73b-13e6-4b14-8bde-683b4f74a16a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:17.560156 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e725c73b-13e6-4b14-8bde-683b4f74a16a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:17.667199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1225/2474] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:07:17 2022] POST /identity/v3/users => generated 289 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:07:17.668773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe9646-4fbf-4f81-a59f-55bd78c2ed8c admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:17.680330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe9646-4fbf-4f81-a59f-55bd78c2ed8c admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:17.686797 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396983f9-fd3e-496b-a324-2dfd052ee63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:17.687550 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396983f9-fd3e-496b-a324-2dfd052ee63e admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:17.688091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396983f9-fd3e-496b-a324-2dfd052ee63e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:17.688464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396983f9-fd3e-496b-a324-2dfd052ee63e admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:17.689274 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396983f9-fd3e-496b-a324-2dfd052ee63e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:17.691399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396983f9-fd3e-496b-a324-2dfd052ee63e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:17.706704 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1250/2475] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:07:17.710460 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725c73b-13e6-4b14-8bde-683b4f74a16a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:17.722597 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725c73b-13e6-4b14-8bde-683b4f74a16a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:17.728193 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177a34c-616c-41bc-a7ba-6f18d9bec29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:17.728852 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177a34c-616c-41bc-a7ba-6f18d9bec29e admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:17.729153 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177a34c-616c-41bc-a7ba-6f18d9bec29e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:17.729426 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177a34c-616c-41bc-a7ba-6f18d9bec29e admin admin] PATH_INFO: `/v3/projects/331d2fe303964567abbade42dc30d760/users/e7ce0eea0a1a40a7b42fe3104e48fc05/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:17.749156 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177a34c-616c-41bc-a7ba-6f18d9bec29e admin admin] RBAC: Authorizing `identity:create_grant(project_id=331d2fe303964567abbade42dc30d760, user_id=e7ce0eea0a1a40a7b42fe3104e48fc05, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:17.751973 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177a34c-616c-41bc-a7ba-6f18d9bec29e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:17.829299 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1226/2476] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:07:17 2022] PUT /identity/v3/projects/331d2fe303964567abbade42dc30d760/users/e7ce0eea0a1a40a7b42fe3104e48fc05/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:17.832984 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396983f9-fd3e-496b-a324-2dfd052ee63e admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:17.846016 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396983f9-fd3e-496b-a324-2dfd052ee63e admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:17.918769 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f9da7c-19c4-4ecf-b4c0-949690491d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:17.919564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9da7c-19c4-4ecf-b4c0-949690491d23 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:17.920121 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9da7c-19c4-4ecf-b4c0-949690491d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:17.920496 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9da7c-19c4-4ecf-b4c0-949690491d23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:17.921233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f9da7c-19c4-4ecf-b4c0-949690491d23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:17.922972 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f9da7c-19c4-4ecf-b4c0-949690491d23 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:17.925365 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:17.940352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1251/2477] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:07:17.941689 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177a34c-616c-41bc-a7ba-6f18d9bec29e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:17.963153 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177a34c-616c-41bc-a7ba-6f18d9bec29e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:17.968919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94fca5-4657-4eef-b87b-ae327067bf4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:17.971299 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94fca5-4657-4eef-b87b-ae327067bf4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:17.983591 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94fca5-4657-4eef-b87b-ae327067bf4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:17.984023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94fca5-4657-4eef-b87b-ae327067bf4d admin admin] PATH_INFO: `/v3/projects/331d2fe303964567abbade42dc30d760/users/e7ce0eea0a1a40a7b42fe3104e48fc05/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:17.987270 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd94fca5-4657-4eef-b87b-ae327067bf4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=331d2fe303964567abbade42dc30d760, user_id=e7ce0eea0a1a40a7b42fe3104e48fc05, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:17.993580 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd94fca5-4657-4eef-b87b-ae327067bf4d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:18.019350 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1227/2478] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:07:17 2022] PUT /identity/v3/projects/331d2fe303964567abbade42dc30d760/users/e7ce0eea0a1a40a7b42fe3104e48fc05/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:18.029842 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cd3e5-66ad-4153-b44a-96f90893dbfb None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:18.030129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cd3e5-66ad-4153-b44a-96f90893dbfb None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:18.030599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cd3e5-66ad-4153-b44a-96f90893dbfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:18.064077 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-ba1cd3e5-66ad-4153-b44a-96f90893dbfb None None] MFA Rules not processed for user `e7ce0eea0a1a40a7b42fe3104e48fc05`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:18.065339 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ba1cd3e5-66ad-4153-b44a-96f90893dbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:18.144806 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1252/2479] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:18.160823 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94fca5-4657-4eef-b87b-ae327067bf4d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:18.163313 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fd94fca5-4657-4eef-b87b-ae327067bf4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:18.237679 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94fca5-4657-4eef-b87b-ae327067bf4d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:18.242273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc305ed8-5e89-4d31-85f6-0370e597b090 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:18.242918 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc305ed8-5e89-4d31-85f6-0370e597b090 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:18.243194 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc305ed8-5e89-4d31-85f6-0370e597b090 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:18.243482 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc305ed8-5e89-4d31-85f6-0370e597b090 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:18.253002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc305ed8-5e89-4d31-85f6-0370e597b090 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:18.254683 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc305ed8-5e89-4d31-85f6-0370e597b090 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:18.265446 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1228/2480] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:07:18 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:07:18.582379 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1cd3e5-66ad-4153-b44a-96f90893dbfb None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:18.583953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ba1cd3e5-66ad-4153-b44a-96f90893dbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:18.598149 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1cd3e5-66ad-4153-b44a-96f90893dbfb None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:18.602074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e51e21-249a-4ffe-9625-c71e1093ae5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:18.602717 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e51e21-249a-4ffe-9625-c71e1093ae5c service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:18.602983 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e51e21-249a-4ffe-9625-c71e1093ae5c service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:18.603298 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e51e21-249a-4ffe-9625-c71e1093ae5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:18.685653 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e51e21-249a-4ffe-9625-c71e1093ae5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:18.687140 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e51e21-249a-4ffe-9625-c71e1093ae5c service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:18.698350 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc305ed8-5e89-4d31-85f6-0370e597b090 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:18.700125 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-cc305ed8-5e89-4d31-85f6-0370e597b090 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:18.702818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1253/2481] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:07:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:07:18.744812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:18.759472 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc305ed8-5e89-4d31-85f6-0370e597b090 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:18.763082 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681080b-dea4-442d-bf09-4750f4f7d73e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:18.763666 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681080b-dea4-442d-bf09-4750f4f7d73e service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:18.763939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681080b-dea4-442d-bf09-4750f4f7d73e service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:18.764205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681080b-dea4-442d-bf09-4750f4f7d73e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:18.765929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f681080b-dea4-442d-bf09-4750f4f7d73e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:18.828725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f681080b-dea4-442d-bf09-4750f4f7d73e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:18.830160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f681080b-dea4-442d-bf09-4750f4f7d73e service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:18.839604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1229/2482] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:07:18 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:07:21.080531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e51e21-249a-4ffe-9625-c71e1093ae5c service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:21.082056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-21e51e21-249a-4ffe-9625-c71e1093ae5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:21.160650 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e51e21-249a-4ffe-9625-c71e1093ae5c service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:21.168369 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9053a3a-7211-4082-af37-fca0b457a04d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:21.169002 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9053a3a-7211-4082-af37-fca0b457a04d service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:21.169281 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9053a3a-7211-4082-af37-fca0b457a04d service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:21.169554 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9053a3a-7211-4082-af37-fca0b457a04d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:21.171421 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c9053a3a-7211-4082-af37-fca0b457a04d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:21.185581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9053a3a-7211-4082-af37-fca0b457a04d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:21.186965 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9053a3a-7211-4082-af37-fca0b457a04d service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:21.204098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1254/2483] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:07:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:07:22.127608 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681080b-dea4-442d-bf09-4750f4f7d73e service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:22.222270 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681080b-dea4-442d-bf09-4750f4f7d73e service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:22.229882 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa99120-4d8b-4999-8d27-41162aa184ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:22.230528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa99120-4d8b-4999-8d27-41162aa184ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:22.230848 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa99120-4d8b-4999-8d27-41162aa184ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:22.231122 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa99120-4d8b-4999-8d27-41162aa184ae admin admin] PATH_INFO: `/v3/users/33f37811ea854ebf8f77e54e258f7691` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:22.232565 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa99120-4d8b-4999-8d27-41162aa184ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=33f37811ea854ebf8f77e54e258f7691)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:22.234832 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa99120-4d8b-4999-8d27-41162aa184ae admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:22.459606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-eaa99120-4d8b-4999-8d27-41162aa184ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33f37811ea854ebf8f77e54e258f7691'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:22.466971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-eaa99120-4d8b-4999-8d27-41162aa184ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33f37811ea854ebf8f77e54e258f7691'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:22.485014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-eaa99120-4d8b-4999-8d27-41162aa184ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33f37811ea854ebf8f77e54e258f7691'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:22.559025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-eaa99120-4d8b-4999-8d27-41162aa184ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33f37811ea854ebf8f77e54e258f7691'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:22.560266 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-eaa99120-4d8b-4999-8d27-41162aa184ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33f37811ea854ebf8f77e54e258f7691'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:22.566969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1230/2484] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:07:22 2022] DELETE /identity/v3/users/33f37811ea854ebf8f77e54e258f7691 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:22.572471 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a2190-2c4f-4cdb-a88b-f46062b2c50e None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:22.573452 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a2190-2c4f-4cdb-a88b-f46062b2c50e None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:22.575182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a2190-2c4f-4cdb-a88b-f46062b2c50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:22.589171 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:22.595449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a80a2190-2c4f-4cdb-a88b-f46062b2c50e None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:22.596653 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a80a2190-2c4f-4cdb-a88b-f46062b2c50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:22.659762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1255/2485] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:22.670360 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa99120-4d8b-4999-8d27-41162aa184ae admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:22.671827 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-eaa99120-4d8b-4999-8d27-41162aa184ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:22.682055 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:22.732827 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa99120-4d8b-4999-8d27-41162aa184ae admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:22.738216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb76f4-ea82-4bd8-beb3-75167cd11fbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:22.738850 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb76f4-ea82-4bd8-beb3-75167cd11fbd service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:22.739124 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb76f4-ea82-4bd8-beb3-75167cd11fbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:22.739394 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb76f4-ea82-4bd8-beb3-75167cd11fbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:22.749457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb76f4-ea82-4bd8-beb3-75167cd11fbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:22.753825 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb76f4-ea82-4bd8-beb3-75167cd11fbd service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:22.761586 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1231/2486] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:07:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:07:23.241307 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a2190-2c4f-4cdb-a88b-f46062b2c50e None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:23.242793 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a80a2190-2c4f-4cdb-a88b-f46062b2c50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:23.257233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a2190-2c4f-4cdb-a88b-f46062b2c50e None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:23.265089 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13118cab-1919-4dc7-be12-c8f5e3f34c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:23.265781 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13118cab-1919-4dc7-be12-c8f5e3f34c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:23.266055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13118cab-1919-4dc7-be12-c8f5e3f34c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:23.266327 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13118cab-1919-4dc7-be12-c8f5e3f34c55 admin admin] PATH_INFO: `/v3/projects/4e181a30cb154859a7a283dffb8d4c2c` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:23.271520 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13118cab-1919-4dc7-be12-c8f5e3f34c55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e181a30cb154859a7a283dffb8d4c2c)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:23.271520 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13118cab-1919-4dc7-be12-c8f5e3f34c55 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:23.405305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-13118cab-1919-4dc7-be12-c8f5e3f34c55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e181a30cb154859a7a283dffb8d4c2c', 'request_id': 'req-13118cab-1919-4dc7-be12-c8f5e3f34c55'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:23.478944 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-13118cab-1919-4dc7-be12-c8f5e3f34c55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e181a30cb154859a7a283dffb8d4c2c', 'request_id': 'req-13118cab-1919-4dc7-be12-c8f5e3f34c55'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:23.485972 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-13118cab-1919-4dc7-be12-c8f5e3f34c55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e181a30cb154859a7a283dffb8d4c2c', 'request_id': 'req-13118cab-1919-4dc7-be12-c8f5e3f34c55'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:23.497748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-13118cab-1919-4dc7-be12-c8f5e3f34c55 admin admin] The token cache is being invalidate because project 4e181a30cb154859a7a283dffb8d4c2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:07:23.498040 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-13118cab-1919-4dc7-be12-c8f5e3f34c55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:23.498736 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-13118cab-1919-4dc7-be12-c8f5e3f34c55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:23.500280 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1256/2487] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:07:23 2022] DELETE /identity/v3/projects/4e181a30cb154859a7a283dffb8d4c2c => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:23.504321 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb76f4-ea82-4bd8-beb3-75167cd11fbd service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:23.515819 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb76f4-ea82-4bd8-beb3-75167cd11fbd service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:23.578729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b98b3-9c04-4403-a468-1f0000f6f66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:23.579383 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b98b3-9c04-4403-a468-1f0000f6f66a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:23.579655 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b98b3-9c04-4403-a468-1f0000f6f66a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:23.579920 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b98b3-9c04-4403-a468-1f0000f6f66a admin admin] PATH_INFO: `/v3/users/e7ce0eea0a1a40a7b42fe3104e48fc05` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:23.581067 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b98b3-9c04-4403-a468-1f0000f6f66a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7ce0eea0a1a40a7b42fe3104e48fc05)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:23.583049 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b98b3-9c04-4403-a468-1f0000f6f66a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:23.781048 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6f6b98b3-9c04-4403-a468-1f0000f6f66a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7ce0eea0a1a40a7b42fe3104e48fc05'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:23.788668 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6f6b98b3-9c04-4403-a468-1f0000f6f66a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7ce0eea0a1a40a7b42fe3104e48fc05'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:23.801282 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6f6b98b3-9c04-4403-a468-1f0000f6f66a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7ce0eea0a1a40a7b42fe3104e48fc05'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:23.885165 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6f6b98b3-9c04-4403-a468-1f0000f6f66a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7ce0eea0a1a40a7b42fe3104e48fc05'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:23.885787 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6f6b98b3-9c04-4403-a468-1f0000f6f66a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7ce0eea0a1a40a7b42fe3104e48fc05'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:23.892153 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1232/2488] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:07:23 2022] DELETE /identity/v3/users/e7ce0eea0a1a40a7b42fe3104e48fc05 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:24.362137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13118cab-1919-4dc7-be12-c8f5e3f34c55 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:24.363788 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-13118cab-1919-4dc7-be12-c8f5e3f34c55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:24.379393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13118cab-1919-4dc7-be12-c8f5e3f34c55 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:24.454986 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda7b52-49d9-40d8-a68c-3dd87b935ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:24.456066 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda7b52-49d9-40d8-a68c-3dd87b935ac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:24.456348 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda7b52-49d9-40d8-a68c-3dd87b935ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:24.456623 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda7b52-49d9-40d8-a68c-3dd87b935ac4 admin admin] PATH_INFO: `/v3/projects/331d2fe303964567abbade42dc30d760` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:24.461633 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda7b52-49d9-40d8-a68c-3dd87b935ac4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=331d2fe303964567abbade42dc30d760)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:24.464574 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda7b52-49d9-40d8-a68c-3dd87b935ac4 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:24.571947 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:24.600726 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-9bda7b52-49d9-40d8-a68c-3dd87b935ac4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '331d2fe303964567abbade42dc30d760', 'request_id': 'req-9bda7b52-49d9-40d8-a68c-3dd87b935ac4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:24.651101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-9bda7b52-49d9-40d8-a68c-3dd87b935ac4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '331d2fe303964567abbade42dc30d760', 'request_id': 'req-9bda7b52-49d9-40d8-a68c-3dd87b935ac4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:24.657646 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-9bda7b52-49d9-40d8-a68c-3dd87b935ac4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '331d2fe303964567abbade42dc30d760', 'request_id': 'req-9bda7b52-49d9-40d8-a68c-3dd87b935ac4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:24.671974 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-9bda7b52-49d9-40d8-a68c-3dd87b935ac4 admin admin] The token cache is being invalidate because project 331d2fe303964567abbade42dc30d760 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:07:24.672245 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-9bda7b52-49d9-40d8-a68c-3dd87b935ac4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:24.672951 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-9bda7b52-49d9-40d8-a68c-3dd87b935ac4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:24.678494 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1257/2489] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:07:24 2022] DELETE /identity/v3/projects/331d2fe303964567abbade42dc30d760 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:24.680820 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3f907-e25c-44e5-af5e-ac1e09ba800a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:24.681726 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3f907-e25c-44e5-af5e-ac1e09ba800a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:24.682231 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3f907-e25c-44e5-af5e-ac1e09ba800a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:24.710772 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-80a3f907-e25c-44e5-af5e-ac1e09ba800a None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:24.712635 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-80a3f907-e25c-44e5-af5e-ac1e09ba800a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:24.760272 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:24.814442 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1233/2490] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:24.823007 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96033a7-b6b6-4f85-8717-6792756bde23 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:24.823445 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96033a7-b6b6-4f85-8717-6792756bde23 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:24.823745 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96033a7-b6b6-4f85-8717-6792756bde23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:24.847851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-d96033a7-b6b6-4f85-8717-6792756bde23 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:24.849609 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-d96033a7-b6b6-4f85-8717-6792756bde23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:24.863069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1258/2491] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:24.868224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3f907-e25c-44e5-af5e-ac1e09ba800a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:24.878961 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3f907-e25c-44e5-af5e-ac1e09ba800a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:24.883810 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae3660-80ae-4e9e-a92b-c456dba4f4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:24.884417 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae3660-80ae-4e9e-a92b-c456dba4f4fc admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:24.886614 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae3660-80ae-4e9e-a92b-c456dba4f4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:24.886614 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae3660-80ae-4e9e-a92b-c456dba4f4fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:24.886614 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae3660-80ae-4e9e-a92b-c456dba4f4fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:07:24.886614 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae3660-80ae-4e9e-a92b-c456dba4f4fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:24.887839 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae3660-80ae-4e9e-a92b-c456dba4f4fc admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:24.904855 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1234/2492] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:07:24.906091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96033a7-b6b6-4f85-8717-6792756bde23 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:24.915477 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96033a7-b6b6-4f85-8717-6792756bde23 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:24.920584 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33e704-0471-46fb-8827-01522860ead2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:24.921146 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33e704-0471-46fb-8827-01522860ead2 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:24.921471 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33e704-0471-46fb-8827-01522860ead2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:24.921749 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33e704-0471-46fb-8827-01522860ead2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:24.922408 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb33e704-0471-46fb-8827-01522860ead2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:24.924349 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb33e704-0471-46fb-8827-01522860ead2 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:25.005484 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1259/2493] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:07:24 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:07:25.007202 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae3660-80ae-4e9e-a92b-c456dba4f4fc admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:25.019011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae3660-80ae-4e9e-a92b-c456dba4f4fc admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:25.085087 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc455e-71b5-4be5-a86b-3eb43fbb1b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:25.085716 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc455e-71b5-4be5-a86b-3eb43fbb1b4e admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:25.086297 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc455e-71b5-4be5-a86b-3eb43fbb1b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:25.087008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc455e-71b5-4be5-a86b-3eb43fbb1b4e admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:25.087721 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc455e-71b5-4be5-a86b-3eb43fbb1b4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:25.089675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc455e-71b5-4be5-a86b-3eb43fbb1b4e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:25.204045 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1235/2494] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:07:25 2022] POST /identity/v3/users => generated 292 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:07:25.204867 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33e704-0471-46fb-8827-01522860ead2 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:25.216700 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33e704-0471-46fb-8827-01522860ead2 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:25.222179 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217815f9-b471-49c9-baae-da1d7a809443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:25.222921 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217815f9-b471-49c9-baae-da1d7a809443 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:25.223139 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217815f9-b471-49c9-baae-da1d7a809443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:25.223415 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217815f9-b471-49c9-baae-da1d7a809443 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:25.224138 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217815f9-b471-49c9-baae-da1d7a809443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:25.225726 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217815f9-b471-49c9-baae-da1d7a809443 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:25.238154 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1260/2495] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:07:25.240259 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc455e-71b5-4be5-a86b-3eb43fbb1b4e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:25.250246 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc455e-71b5-4be5-a86b-3eb43fbb1b4e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:25.254972 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d0ea9-8a2e-418e-9140-f911ce04ef03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:25.256100 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d0ea9-8a2e-418e-9140-f911ce04ef03 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:25.256371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d0ea9-8a2e-418e-9140-f911ce04ef03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:25.256641 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d0ea9-8a2e-418e-9140-f911ce04ef03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:25.257302 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392d0ea9-8a2e-418e-9140-f911ce04ef03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:25.258910 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392d0ea9-8a2e-418e-9140-f911ce04ef03 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:25.269319 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1236/2496] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:07:25.274724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217815f9-b471-49c9-baae-da1d7a809443 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:25.284365 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217815f9-b471-49c9-baae-da1d7a809443 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:25.290355 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d59ee-3a3d-4e55-893d-389ec17800d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:25.291031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d59ee-3a3d-4e55-893d-389ec17800d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:25.291307 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d59ee-3a3d-4e55-893d-389ec17800d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:25.291577 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d59ee-3a3d-4e55-893d-389ec17800d1 admin admin] PATH_INFO: `/v3/projects/f86daf88227b4dcaa11e0226e4cd7177/users/273de641d38a40ebb50832ff800c7999/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:25.306849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d59ee-3a3d-4e55-893d-389ec17800d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f86daf88227b4dcaa11e0226e4cd7177, user_id=273de641d38a40ebb50832ff800c7999, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:25.309506 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d59ee-3a3d-4e55-893d-389ec17800d1 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:25.399411 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1261/2497] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:07:25 2022] PUT /identity/v3/projects/f86daf88227b4dcaa11e0226e4cd7177/users/273de641d38a40ebb50832ff800c7999/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:25.410712 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69aef4-7a3e-44ac-abc2-a54d603ea620 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:25.411198 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69aef4-7a3e-44ac-abc2-a54d603ea620 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:25.411503 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69aef4-7a3e-44ac-abc2-a54d603ea620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:25.454026 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-cd69aef4-7a3e-44ac-abc2-a54d603ea620 None None] MFA Rules not processed for user `273de641d38a40ebb50832ff800c7999`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:25.455196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-cd69aef4-7a3e-44ac-abc2-a54d603ea620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:25.531067 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1237/2498] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:25.535571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d59ee-3a3d-4e55-893d-389ec17800d1 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:25.606915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d59ee-3a3d-4e55-893d-389ec17800d1 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:25.611729 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b160b2-7ef4-44a5-9d4b-d4252eb5c128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:25.613261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b160b2-7ef4-44a5-9d4b-d4252eb5c128 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:25.613261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b160b2-7ef4-44a5-9d4b-d4252eb5c128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:25.613261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b160b2-7ef4-44a5-9d4b-d4252eb5c128 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:25.614126 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b160b2-7ef4-44a5-9d4b-d4252eb5c128 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:25.616555 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b160b2-7ef4-44a5-9d4b-d4252eb5c128 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:25.719723 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1262/2499] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:07:25 2022] POST /identity/v3/projects => generated 383 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:07:25.723556 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69aef4-7a3e-44ac-abc2-a54d603ea620 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:25.735612 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69aef4-7a3e-44ac-abc2-a54d603ea620 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:25.759773 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:25.811939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c44487-1983-4a3d-96ac-7be58bad458d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:25.812551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c44487-1983-4a3d-96ac-7be58bad458d admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:25.812826 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c44487-1983-4a3d-96ac-7be58bad458d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:25.813097 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c44487-1983-4a3d-96ac-7be58bad458d admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:25.813798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c44487-1983-4a3d-96ac-7be58bad458d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:25.815933 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c44487-1983-4a3d-96ac-7be58bad458d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:25.871105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5497a9b9-8b2e-4546-b56b-e584263a1ef7 None None] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:25.871387 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5497a9b9-8b2e-4546-b56b-e584263a1ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:25.871685 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5497a9b9-8b2e-4546-b56b-e584263a1ef7 None None] PATH_INFO: `/` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:25.873104 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1263/2500] 10.222.0.17 () {54 vars in 921 bytes} [Sun Sep 11 00:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 00:07:25.880557 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a3c1a-ca3a-4c43-8729-daa0dce503e4 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:25.880832 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a3c1a-ca3a-4c43-8729-daa0dce503e4 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:25.881104 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a3c1a-ca3a-4c43-8729-daa0dce503e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:25.904347 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1238/2501] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:07:25 2022] POST /identity/v3/users => generated 298 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:07:25.904554 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-479a3c1a-ca3a-4c43-8729-daa0dce503e4 None None] MFA Rules not processed for user `636bfafb693044d283530063047fc0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:25.906065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c44487-1983-4a3d-96ac-7be58bad458d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:25.906256 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-479a3c1a-ca3a-4c43-8729-daa0dce503e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:25.944766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c44487-1983-4a3d-96ac-7be58bad458d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:25.951017 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4abc1d2-0e6e-43c8-9b9d-060fda815291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:25.951767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4abc1d2-0e6e-43c8-9b9d-060fda815291 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:25.953199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4abc1d2-0e6e-43c8-9b9d-060fda815291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:25.953199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4abc1d2-0e6e-43c8-9b9d-060fda815291 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:25.953199 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4abc1d2-0e6e-43c8-9b9d-060fda815291 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:25.955542 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4abc1d2-0e6e-43c8-9b9d-060fda815291 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:25.962890 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:25.975714 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1239/2502] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:07:25.985249 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4abc1d2-0e6e-43c8-9b9d-060fda815291 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:25.987004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4abc1d2-0e6e-43c8-9b9d-060fda815291 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:25.994713 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa74f445-e5b7-4ae7-8b39-d1b91e8275ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:25.995527 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74f445-e5b7-4ae7-8b39-d1b91e8275ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:25.995959 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74f445-e5b7-4ae7-8b39-d1b91e8275ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:25.996239 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74f445-e5b7-4ae7-8b39-d1b91e8275ed admin admin] PATH_INFO: `/v3/projects/99a19b9876a7445d918a7531057f745b/users/4b6a31e42eda4232952d0efb8d0f9ef3/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:26.013711 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa74f445-e5b7-4ae7-8b39-d1b91e8275ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=99a19b9876a7445d918a7531057f745b, user_id=4b6a31e42eda4232952d0efb8d0f9ef3, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:26.019900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa74f445-e5b7-4ae7-8b39-d1b91e8275ed admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:26.023278 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1264/2503] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Sep 11 00:07:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:07:26.035549 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a3c1a-ca3a-4c43-8729-daa0dce503e4 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:26.047015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-479a3c1a-ca3a-4c43-8729-daa0dce503e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:26.089219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1240/2504] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:07:25 2022] PUT /identity/v3/projects/99a19b9876a7445d918a7531057f745b/users/4b6a31e42eda4232952d0efb8d0f9ef3/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:26.095845 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa74f445-e5b7-4ae7-8b39-d1b91e8275ed admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:26.115398 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a3c1a-ca3a-4c43-8729-daa0dce503e4 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:26.178424 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9db36b-383d-4861-a29c-4f4fee3ca85f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb18199dc8b42589184042556e98983', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:26.179142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9db36b-383d-4861-a29c-4f4fee3ca85f service glance] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:26.179283 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa74f445-e5b7-4ae7-8b39-d1b91e8275ed admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:26.179463 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9db36b-383d-4861-a29c-4f4fee3ca85f service glance] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:26.179876 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9db36b-383d-4861-a29c-4f4fee3ca85f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:26.189648 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8f970-403b-43e5-a67a-3a31c45adf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:26.190307 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8f970-403b-43e5-a67a-3a31c45adf66 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:26.190740 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8f970-403b-43e5-a67a-3a31c45adf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:26.191014 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8f970-403b-43e5-a67a-3a31c45adf66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:26.191686 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b8f970-403b-43e5-a67a-3a31c45adf66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:26.193798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b8f970-403b-43e5-a67a-3a31c45adf66 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:26.205443 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1241/2505] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:07:26.210330 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8f970-403b-43e5-a67a-3a31c45adf66 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:26.220491 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8f970-403b-43e5-a67a-3a31c45adf66 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:26.225494 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12216469-b843-4c0c-a0a1-fd444bef31d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:26.226073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12216469-b843-4c0c-a0a1-fd444bef31d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:26.226343 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12216469-b843-4c0c-a0a1-fd444bef31d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:26.226615 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12216469-b843-4c0c-a0a1-fd444bef31d6 admin admin] PATH_INFO: `/v3/projects/99a19b9876a7445d918a7531057f745b/users/4b6a31e42eda4232952d0efb8d0f9ef3/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:26.228558 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12216469-b843-4c0c-a0a1-fd444bef31d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99a19b9876a7445d918a7531057f745b, user_id=4b6a31e42eda4232952d0efb8d0f9ef3, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:26.231375 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12216469-b843-4c0c-a0a1-fd444bef31d6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:26.245274 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1242/2506] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:07:26 2022] PUT /identity/v3/projects/99a19b9876a7445d918a7531057f745b/users/4b6a31e42eda4232952d0efb8d0f9ef3/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:07:26.254359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b343e1-96f6-47d8-a8f9-0cf86fc55f21 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:26.254635 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b343e1-96f6-47d8-a8f9-0cf86fc55f21 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:26.254954 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b343e1-96f6-47d8-a8f9-0cf86fc55f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:26.284959 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9db36b-383d-4861-a29c-4f4fee3ca85f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:26.286556 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9db36b-383d-4861-a29c-4f4fee3ca85f service glance] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:26.299339 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f3b343e1-96f6-47d8-a8f9-0cf86fc55f21 None None] MFA Rules not processed for user `4b6a31e42eda4232952d0efb8d0f9ef3`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:26.303423 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1265/2507] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:07:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:07:26.303567 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f3b343e1-96f6-47d8-a8f9-0cf86fc55f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:26.359154 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:26.379149 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1243/2508] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:07:26.390649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9db36b-383d-4861-a29c-4f4fee3ca85f service glance] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:26.392098 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ae9db36b-383d-4861-a29c-4f4fee3ca85f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:26.469925 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ff6fb-0c26-4e9f-82d5-baf64481e299 None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:26.470342 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ff6fb-0c26-4e9f-82d5-baf64481e299 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:26.470743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ff6fb-0c26-4e9f-82d5-baf64481e299 None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:26.472170 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1244/2509] 10.222.0.17 () {54 vars in 921 bytes} [Sun Sep 11 00:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 00:07:26.478012 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309ca04-51b9-4add-b577-7d96c8cb5259 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:26.478266 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309ca04-51b9-4add-b577-7d96c8cb5259 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:26.478551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309ca04-51b9-4add-b577-7d96c8cb5259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:26.517625 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5309ca04-51b9-4add-b577-7d96c8cb5259 None None] MFA Rules not processed for user `53e79d9fb2cd452d8d999a84b6425959`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:26.520012 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5309ca04-51b9-4add-b577-7d96c8cb5259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:26.551529 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:26.581504 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1245/2510] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Sep 11 00:07:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:07:26.581739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9db36b-383d-4861-a29c-4f4fee3ca85f service glance] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:26.593898 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309ca04-51b9-4add-b577-7d96c8cb5259 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:26.595286 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5309ca04-51b9-4add-b577-7d96c8cb5259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:26.606255 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309ca04-51b9-4add-b577-7d96c8cb5259 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:26.610400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e5e2c-6d9b-48d1-81d8-14225d972e43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:26.611117 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e5e2c-6d9b-48d1-81d8-14225d972e43 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:26.611392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e5e2c-6d9b-48d1-81d8-14225d972e43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:26.611657 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e5e2c-6d9b-48d1-81d8-14225d972e43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:26.623415 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e5e2c-6d9b-48d1-81d8-14225d972e43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:26.624993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e5e2c-6d9b-48d1-81d8-14225d972e43 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:26.634391 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1246/2511] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:07:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:07:26.636243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bcf870-44c9-4d9b-9db9-a16785c3966e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:26.637225 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcf870-44c9-4d9b-9db9-a16785c3966e service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:26.637876 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcf870-44c9-4d9b-9db9-a16785c3966e service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:26.638370 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcf870-44c9-4d9b-9db9-a16785c3966e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:26.702910 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bcf870-44c9-4d9b-9db9-a16785c3966e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:26.704429 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bcf870-44c9-4d9b-9db9-a16785c3966e service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:26.714376 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1266/2512] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:07:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:07:26.728920 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c19251-d718-48ed-a0bd-5bbea0cb8c8f None None] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:26.729364 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c19251-d718-48ed-a0bd-5bbea0cb8c8f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:26.729617 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c19251-d718-48ed-a0bd-5bbea0cb8c8f None None] PATH_INFO: `/` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:26.731232 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1247/2513] 10.222.0.17 () {54 vars in 921 bytes} [Sun Sep 11 00:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:07:26.736989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014d667-d5c9-4aa7-b689-845394dd7997 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:26.737362 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014d667-d5c9-4aa7-b689-845394dd7997 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:26.737700 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014d667-d5c9-4aa7-b689-845394dd7997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:26.758096 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-2014d667-d5c9-4aa7-b689-845394dd7997 None None] MFA Rules not processed for user `636bfafb693044d283530063047fc0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:26.759348 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2014d667-d5c9-4aa7-b689-845394dd7997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:26.830598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1267/2514] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Sep 11 00:07:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:26.841801 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c19251-d718-48ed-a0bd-5bbea0cb8c8f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:26.855190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c19251-d718-48ed-a0bd-5bbea0cb8c8f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:26.869170 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99436963-4879-4290-9b43-c2be9928c447 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:26.869770 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99436963-4879-4290-9b43-c2be9928c447 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:26.870041 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99436963-4879-4290-9b43-c2be9928c447 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:26.870305 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99436963-4879-4290-9b43-c2be9928c447 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:26.881229 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99436963-4879-4290-9b43-c2be9928c447 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:26.882818 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99436963-4879-4290-9b43-c2be9928c447 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:26.894008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1248/2515] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:07:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:07:29.785836 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c897a-68a7-4109-b840-617b5778d23f None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:29.786300 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c897a-68a7-4109-b840-617b5778d23f None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:29.786395 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c897a-68a7-4109-b840-617b5778d23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:29.825681 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-d15c897a-68a7-4109-b840-617b5778d23f None None] MFA Rules not processed for user `b0a879085a194b0687e1cdcbafd48033`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:29.827630 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-d15c897a-68a7-4109-b840-617b5778d23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:29.839577 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:29.903870 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1268/2516] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Sep 11 00:07:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:29.927300 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99436963-4879-4290-9b43-c2be9928c447 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:29.929211 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-99436963-4879-4290-9b43-c2be9928c447 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:30.029640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99436963-4879-4290-9b43-c2be9928c447 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:30.035067 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c394cf7-074a-47a5-a259-623be1594acf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '564e65585e9543e286b6041af870cf41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:30.035663 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c394cf7-074a-47a5-a259-623be1594acf service swift] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:30.035939 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c394cf7-074a-47a5-a259-623be1594acf service swift] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:30.036209 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c394cf7-074a-47a5-a259-623be1594acf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:30.044747 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c394cf7-074a-47a5-a259-623be1594acf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:30.046320 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c394cf7-074a-47a5-a259-623be1594acf service swift] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:30.055192 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1249/2517] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Sep 11 00:07:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 00:07:32.102130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c897a-68a7-4109-b840-617b5778d23f None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:32.122406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c897a-68a7-4109-b840-617b5778d23f None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:32.126495 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075aa555-5ef7-41cc-86a0-13f6850de007 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:32.127173 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075aa555-5ef7-41cc-86a0-13f6850de007 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:32.127450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075aa555-5ef7-41cc-86a0-13f6850de007 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:32.127723 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075aa555-5ef7-41cc-86a0-13f6850de007 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:32.137618 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075aa555-5ef7-41cc-86a0-13f6850de007 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:32.139087 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075aa555-5ef7-41cc-86a0-13f6850de007 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:32.149022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1269/2518] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:07:32 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:07:39.259751 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c394cf7-074a-47a5-a259-623be1594acf service swift] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:39.289312 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:39.340013 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c394cf7-074a-47a5-a259-623be1594acf service swift] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:39.344417 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e748bed-e109-4b1b-bc49-c2376a1f94b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:39.345029 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e748bed-e109-4b1b-bc49-c2376a1f94b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:39.345302 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e748bed-e109-4b1b-bc49-c2376a1f94b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:39.345566 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e748bed-e109-4b1b-bc49-c2376a1f94b6 admin admin] PATH_INFO: `/v3/users/273de641d38a40ebb50832ff800c7999` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:39.346645 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e748bed-e109-4b1b-bc49-c2376a1f94b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=273de641d38a40ebb50832ff800c7999)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:39.348583 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e748bed-e109-4b1b-bc49-c2376a1f94b6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:39.504094 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9e748bed-e109-4b1b-bc49-c2376a1f94b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '273de641d38a40ebb50832ff800c7999'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:39.511457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9e748bed-e109-4b1b-bc49-c2376a1f94b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '273de641d38a40ebb50832ff800c7999'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:39.523838 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9e748bed-e109-4b1b-bc49-c2376a1f94b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '273de641d38a40ebb50832ff800c7999'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:39.604323 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9e748bed-e109-4b1b-bc49-c2376a1f94b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '273de641d38a40ebb50832ff800c7999'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:39.604858 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9e748bed-e109-4b1b-bc49-c2376a1f94b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '273de641d38a40ebb50832ff800c7999'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:39.610931 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1250/2519] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:07:39 2022] DELETE /identity/v3/users/273de641d38a40ebb50832ff800c7999 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:39.616263 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231d2fb-5c58-4dc9-854a-5df71f69e2dc None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:39.616959 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231d2fb-5c58-4dc9-854a-5df71f69e2dc None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:39.616959 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231d2fb-5c58-4dc9-854a-5df71f69e2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:39.655378 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-c231d2fb-5c58-4dc9-854a-5df71f69e2dc None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:39.656563 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c231d2fb-5c58-4dc9-854a-5df71f69e2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:39.716155 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1270/2520] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:39.726528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e748bed-e109-4b1b-bc49-c2376a1f94b6 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:39.728005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9e748bed-e109-4b1b-bc49-c2376a1f94b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:39.807672 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e748bed-e109-4b1b-bc49-c2376a1f94b6 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:39.811356 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5d39f-4643-4bae-a44e-88b37bdbbbcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:39.814449 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5d39f-4643-4bae-a44e-88b37bdbbbcc service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:39.814449 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5d39f-4643-4bae-a44e-88b37bdbbbcc service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:39.814449 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5d39f-4643-4bae-a44e-88b37bdbbbcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:39.821843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf5d39f-4643-4bae-a44e-88b37bdbbbcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:39.823350 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf5d39f-4643-4bae-a44e-88b37bdbbbcc service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:39.833201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1251/2521] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:07:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:07:40.231080 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231d2fb-5c58-4dc9-854a-5df71f69e2dc None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:40.231080 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c231d2fb-5c58-4dc9-854a-5df71f69e2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:40.242911 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231d2fb-5c58-4dc9-854a-5df71f69e2dc None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:40.248876 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a5fac-3483-4256-accc-7036cd29af71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:40.249772 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a5fac-3483-4256-accc-7036cd29af71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:40.250036 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a5fac-3483-4256-accc-7036cd29af71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:40.250303 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a5fac-3483-4256-accc-7036cd29af71 admin admin] PATH_INFO: `/v3/projects/f86daf88227b4dcaa11e0226e4cd7177` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:40.251864 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024a5fac-3483-4256-accc-7036cd29af71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f86daf88227b4dcaa11e0226e4cd7177)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:40.253699 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024a5fac-3483-4256-accc-7036cd29af71 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:40.413962 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-024a5fac-3483-4256-accc-7036cd29af71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f86daf88227b4dcaa11e0226e4cd7177', 'request_id': 'req-024a5fac-3483-4256-accc-7036cd29af71'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:40.484456 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-024a5fac-3483-4256-accc-7036cd29af71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f86daf88227b4dcaa11e0226e4cd7177', 'request_id': 'req-024a5fac-3483-4256-accc-7036cd29af71'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:40.485689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:40.490687 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-024a5fac-3483-4256-accc-7036cd29af71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f86daf88227b4dcaa11e0226e4cd7177', 'request_id': 'req-024a5fac-3483-4256-accc-7036cd29af71'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:40.501728 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-024a5fac-3483-4256-accc-7036cd29af71 admin admin] The token cache is being invalidate because project f86daf88227b4dcaa11e0226e4cd7177 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:07:40.504123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-024a5fac-3483-4256-accc-7036cd29af71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:40.504123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-024a5fac-3483-4256-accc-7036cd29af71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:40.504413 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1271/2522] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:07:40 2022] DELETE /identity/v3/projects/f86daf88227b4dcaa11e0226e4cd7177 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:40.508849 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5d39f-4643-4bae-a44e-88b37bdbbbcc service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:40.518485 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5d39f-4643-4bae-a44e-88b37bdbbbcc service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:40.528105 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:40.576585 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d965e-bb56-49af-9e29-d05e16a764ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:40.577466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d965e-bb56-49af-9e29-d05e16a764ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:40.577743 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d965e-bb56-49af-9e29-d05e16a764ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:40.578012 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d965e-bb56-49af-9e29-d05e16a764ce admin admin] PATH_INFO: `/v3/users/4b6a31e42eda4232952d0efb8d0f9ef3` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:40.579139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d965e-bb56-49af-9e29-d05e16a764ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b6a31e42eda4232952d0efb8d0f9ef3)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:40.581142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d965e-bb56-49af-9e29-d05e16a764ce admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:40.768206 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cd7d965e-bb56-49af-9e29-d05e16a764ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b6a31e42eda4232952d0efb8d0f9ef3'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:40.776074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cd7d965e-bb56-49af-9e29-d05e16a764ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b6a31e42eda4232952d0efb8d0f9ef3'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:40.788055 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cd7d965e-bb56-49af-9e29-d05e16a764ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b6a31e42eda4232952d0efb8d0f9ef3'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:40.890437 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cd7d965e-bb56-49af-9e29-d05e16a764ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b6a31e42eda4232952d0efb8d0f9ef3'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:40.891084 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cd7d965e-bb56-49af-9e29-d05e16a764ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b6a31e42eda4232952d0efb8d0f9ef3'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:40.897946 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1252/2523] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:07:40 2022] DELETE /identity/v3/users/4b6a31e42eda4232952d0efb8d0f9ef3 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:41.312470 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a5fac-3483-4256-accc-7036cd29af71 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:41.312470 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-024a5fac-3483-4256-accc-7036cd29af71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:41.328933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a5fac-3483-4256-accc-7036cd29af71 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:41.389719 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03928260-fc47-49ce-b78d-4b46ed035844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:41.390371 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03928260-fc47-49ce-b78d-4b46ed035844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:41.390643 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03928260-fc47-49ce-b78d-4b46ed035844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:41.390940 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03928260-fc47-49ce-b78d-4b46ed035844 admin admin] PATH_INFO: `/v3/projects/99a19b9876a7445d918a7531057f745b` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:41.392048 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03928260-fc47-49ce-b78d-4b46ed035844 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99a19b9876a7445d918a7531057f745b)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:41.393922 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03928260-fc47-49ce-b78d-4b46ed035844 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:41.501609 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-03928260-fc47-49ce-b78d-4b46ed035844 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99a19b9876a7445d918a7531057f745b', 'request_id': 'req-03928260-fc47-49ce-b78d-4b46ed035844'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:41.559628 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-03928260-fc47-49ce-b78d-4b46ed035844 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99a19b9876a7445d918a7531057f745b', 'request_id': 'req-03928260-fc47-49ce-b78d-4b46ed035844'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:41.568948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-03928260-fc47-49ce-b78d-4b46ed035844 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99a19b9876a7445d918a7531057f745b', 'request_id': 'req-03928260-fc47-49ce-b78d-4b46ed035844'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:41.587801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-03928260-fc47-49ce-b78d-4b46ed035844 admin admin] The token cache is being invalidate because project 99a19b9876a7445d918a7531057f745b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:07:41.588185 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-03928260-fc47-49ce-b78d-4b46ed035844 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:41.589030 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-03928260-fc47-49ce-b78d-4b46ed035844 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:41.590875 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1272/2524] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:07:41 2022] DELETE /identity/v3/projects/99a19b9876a7445d918a7531057f745b => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:41.596851 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e4761-e943-4643-9a8f-40529ed53c52 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:41.597435 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e4761-e943-4643-9a8f-40529ed53c52 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:41.597435 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e4761-e943-4643-9a8f-40529ed53c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:41.672678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-bf6e4761-e943-4643-9a8f-40529ed53c52 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:41.674022 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-bf6e4761-e943-4643-9a8f-40529ed53c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:41.739579 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1253/2525] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:41.749757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3508ac-ba32-493e-b431-bd544db03cf0 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:41.750102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3508ac-ba32-493e-b431-bd544db03cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:41.750436 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3508ac-ba32-493e-b431-bd544db03cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:41.770810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-3e3508ac-ba32-493e-b431-bd544db03cf0 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:41.772043 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3e3508ac-ba32-493e-b431-bd544db03cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:41.780049 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1273/2526] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:41.784357 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e4761-e943-4643-9a8f-40529ed53c52 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:41.795924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e4761-e943-4643-9a8f-40529ed53c52 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:41.800527 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1269cfc-895a-4cdd-855e-1991aa48201b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:41.801124 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1269cfc-895a-4cdd-855e-1991aa48201b admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:41.801405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1269cfc-895a-4cdd-855e-1991aa48201b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:41.801675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1269cfc-895a-4cdd-855e-1991aa48201b admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:41.802316 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1269cfc-895a-4cdd-855e-1991aa48201b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:07:41.803329 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1269cfc-895a-4cdd-855e-1991aa48201b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:41.804862 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1269cfc-895a-4cdd-855e-1991aa48201b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:41.807327 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:41.816975 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1254/2527] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:07:41.825971 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3508ac-ba32-493e-b431-bd544db03cf0 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:41.835175 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3508ac-ba32-493e-b431-bd544db03cf0 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:41.839897 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84008965-8975-4a94-8b3d-3f15cf2e159a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:41.840487 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84008965-8975-4a94-8b3d-3f15cf2e159a admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:41.840760 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84008965-8975-4a94-8b3d-3f15cf2e159a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:41.841027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84008965-8975-4a94-8b3d-3f15cf2e159a admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:41.841646 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84008965-8975-4a94-8b3d-3f15cf2e159a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:41.843640 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84008965-8975-4a94-8b3d-3f15cf2e159a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:41.969994 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1274/2528] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:07:41 2022] POST /identity/v3/projects => generated 385 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:07:41.976009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1269cfc-895a-4cdd-855e-1991aa48201b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:41.982650 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1269cfc-895a-4cdd-855e-1991aa48201b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:42.038973 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933aeb52-61c2-416e-b2d1-3aceb5cff6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:42.039555 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933aeb52-61c2-416e-b2d1-3aceb5cff6b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:42.039854 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933aeb52-61c2-416e-b2d1-3aceb5cff6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:42.040122 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933aeb52-61c2-416e-b2d1-3aceb5cff6b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:42.040746 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933aeb52-61c2-416e-b2d1-3aceb5cff6b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:42.042649 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933aeb52-61c2-416e-b2d1-3aceb5cff6b4 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:42.113028 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1255/2529] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:07:41 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:07:42.117796 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84008965-8975-4a94-8b3d-3f15cf2e159a admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:42.128200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84008965-8975-4a94-8b3d-3f15cf2e159a admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:42.133537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b2b1c-bb41-4a9e-8637-d5f12d07ccb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:42.134128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b2b1c-bb41-4a9e-8637-d5f12d07ccb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:42.134406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b2b1c-bb41-4a9e-8637-d5f12d07ccb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:42.134735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b2b1c-bb41-4a9e-8637-d5f12d07ccb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:42.135373 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b2b1c-bb41-4a9e-8637-d5f12d07ccb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:42.136971 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b2b1c-bb41-4a9e-8637-d5f12d07ccb1 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:42.146846 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1275/2530] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:07:42.154144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933aeb52-61c2-416e-b2d1-3aceb5cff6b4 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:42.160890 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933aeb52-61c2-416e-b2d1-3aceb5cff6b4 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:42.165819 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093ea32-828a-4ad8-a70f-7672cb7564aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:42.166400 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093ea32-828a-4ad8-a70f-7672cb7564aa admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:42.166693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093ea32-828a-4ad8-a70f-7672cb7564aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:42.166968 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093ea32-828a-4ad8-a70f-7672cb7564aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:42.167599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093ea32-828a-4ad8-a70f-7672cb7564aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:42.169229 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093ea32-828a-4ad8-a70f-7672cb7564aa admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:42.179722 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1256/2531] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:07:42.183879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b2b1c-bb41-4a9e-8637-d5f12d07ccb1 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:42.194385 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b2b1c-bb41-4a9e-8637-d5f12d07ccb1 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:42.198926 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e85ab-fbf7-4d8c-b371-57d22ee777ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:42.199820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e85ab-fbf7-4d8c-b371-57d22ee777ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:42.200100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e85ab-fbf7-4d8c-b371-57d22ee777ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:42.200398 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e85ab-fbf7-4d8c-b371-57d22ee777ad admin admin] PATH_INFO: `/v3/projects/1c7c288463494c8090bec970ff88b90a/users/6f2c91dc862049478cb48defcc44e6fa/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:42.215119 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e85ab-fbf7-4d8c-b371-57d22ee777ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c7c288463494c8090bec970ff88b90a, user_id=6f2c91dc862049478cb48defcc44e6fa, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:42.217990 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e85ab-fbf7-4d8c-b371-57d22ee777ad admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:42.296076 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1276/2532] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:07:42 2022] PUT /identity/v3/projects/1c7c288463494c8090bec970ff88b90a/users/6f2c91dc862049478cb48defcc44e6fa/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:42.312498 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d5a3d-0893-424a-b4a3-971db4dab6cd None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:42.312781 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d5a3d-0893-424a-b4a3-971db4dab6cd None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:42.313048 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d5a3d-0893-424a-b4a3-971db4dab6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:42.345235 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-3e9d5a3d-0893-424a-b4a3-971db4dab6cd None None] MFA Rules not processed for user `6f2c91dc862049478cb48defcc44e6fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:42.346458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3e9d5a3d-0893-424a-b4a3-971db4dab6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:42.429814 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1257/2533] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:42.444707 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e85ab-fbf7-4d8c-b371-57d22ee777ad admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:42.446715 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-526e85ab-fbf7-4d8c-b371-57d22ee777ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:42.452254 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:42.520757 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e85ab-fbf7-4d8c-b371-57d22ee777ad admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:42.525282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672eb655-1e57-4b3c-afb0-c40acabdbacf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:42.525879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672eb655-1e57-4b3c-afb0-c40acabdbacf service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:42.526163 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672eb655-1e57-4b3c-afb0-c40acabdbacf service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:42.526428 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672eb655-1e57-4b3c-afb0-c40acabdbacf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:42.534572 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672eb655-1e57-4b3c-afb0-c40acabdbacf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:42.541656 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672eb655-1e57-4b3c-afb0-c40acabdbacf service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:42.546343 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1277/2534] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:07:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:07:42.566316 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d5a3d-0893-424a-b4a3-971db4dab6cd None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:42.628815 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d5a3d-0893-424a-b4a3-971db4dab6cd None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:42.633419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9dbf59-4216-4df6-8d4c-2b53d0d20277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:42.634032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9dbf59-4216-4df6-8d4c-2b53d0d20277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:42.634304 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9dbf59-4216-4df6-8d4c-2b53d0d20277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:42.634576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9dbf59-4216-4df6-8d4c-2b53d0d20277 admin admin] PATH_INFO: `/v3/users/6f2c91dc862049478cb48defcc44e6fa` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:42.635704 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9dbf59-4216-4df6-8d4c-2b53d0d20277 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f2c91dc862049478cb48defcc44e6fa)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:42.637613 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9dbf59-4216-4df6-8d4c-2b53d0d20277 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:42.642259 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:42.841603 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6a9dbf59-4216-4df6-8d4c-2b53d0d20277 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f2c91dc862049478cb48defcc44e6fa'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:42.848523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6a9dbf59-4216-4df6-8d4c-2b53d0d20277 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f2c91dc862049478cb48defcc44e6fa'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:42.860757 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6a9dbf59-4216-4df6-8d4c-2b53d0d20277 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f2c91dc862049478cb48defcc44e6fa'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:42.914436 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6a9dbf59-4216-4df6-8d4c-2b53d0d20277 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f2c91dc862049478cb48defcc44e6fa'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:42.915113 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-6a9dbf59-4216-4df6-8d4c-2b53d0d20277 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f2c91dc862049478cb48defcc44e6fa'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:42.921747 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1258/2535] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:07:42 2022] DELETE /identity/v3/users/6f2c91dc862049478cb48defcc44e6fa => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:42.926343 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455f849-fd91-48d7-951e-1c44d26af686 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:42.926626 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455f849-fd91-48d7-951e-1c44d26af686 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:42.926922 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455f849-fd91-48d7-951e-1c44d26af686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:42.946031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-4455f849-fd91-48d7-951e-1c44d26af686 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:42.950220 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4455f849-fd91-48d7-951e-1c44d26af686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:43.009809 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1278/2536] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:07:43.019602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9dbf59-4216-4df6-8d4c-2b53d0d20277 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:43.021089 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6a9dbf59-4216-4df6-8d4c-2b53d0d20277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:43.092738 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9dbf59-4216-4df6-8d4c-2b53d0d20277 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:43.096481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fdfab-04f1-4813-a735-f1d3667b4f76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:43.097076 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fdfab-04f1-4813-a735-f1d3667b4f76 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:43.097349 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fdfab-04f1-4813-a735-f1d3667b4f76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:43.097614 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fdfab-04f1-4813-a735-f1d3667b4f76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:43.108556 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3fdfab-04f1-4813-a735-f1d3667b4f76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:43.109993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3fdfab-04f1-4813-a735-f1d3667b4f76 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:43.119977 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1259/2537] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:07:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:07:43.561235 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455f849-fd91-48d7-951e-1c44d26af686 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:43.562785 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4455f849-fd91-48d7-951e-1c44d26af686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:43.578146 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455f849-fd91-48d7-951e-1c44d26af686 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:43.583051 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a77dc0-2b2e-4dd5-b5d5-3ec62373aa2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:43.583917 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a77dc0-2b2e-4dd5-b5d5-3ec62373aa2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:43.584388 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a77dc0-2b2e-4dd5-b5d5-3ec62373aa2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:43.584785 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a77dc0-2b2e-4dd5-b5d5-3ec62373aa2e admin admin] PATH_INFO: `/v3/projects/1c7c288463494c8090bec970ff88b90a` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:43.585978 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a77dc0-2b2e-4dd5-b5d5-3ec62373aa2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c7c288463494c8090bec970ff88b90a)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:43.591661 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a77dc0-2b2e-4dd5-b5d5-3ec62373aa2e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:43.690310 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f7a77dc0-2b2e-4dd5-b5d5-3ec62373aa2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c7c288463494c8090bec970ff88b90a', 'request_id': 'req-f7a77dc0-2b2e-4dd5-b5d5-3ec62373aa2e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:43.758895 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f7a77dc0-2b2e-4dd5-b5d5-3ec62373aa2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c7c288463494c8090bec970ff88b90a', 'request_id': 'req-f7a77dc0-2b2e-4dd5-b5d5-3ec62373aa2e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:43.764730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f7a77dc0-2b2e-4dd5-b5d5-3ec62373aa2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c7c288463494c8090bec970ff88b90a', 'request_id': 'req-f7a77dc0-2b2e-4dd5-b5d5-3ec62373aa2e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:43.776390 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f7a77dc0-2b2e-4dd5-b5d5-3ec62373aa2e admin admin] The token cache is being invalidate because project 1c7c288463494c8090bec970ff88b90a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:07:43.776671 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f7a77dc0-2b2e-4dd5-b5d5-3ec62373aa2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:43.777288 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f7a77dc0-2b2e-4dd5-b5d5-3ec62373aa2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:43.778743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1279/2538] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:07:43 2022] DELETE /identity/v3/projects/1c7c288463494c8090bec970ff88b90a => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:43.783672 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236e1f9-8f37-48e3-93c4-ea20d8ae1cf0 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:43.784143 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236e1f9-8f37-48e3-93c4-ea20d8ae1cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:43.784238 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236e1f9-8f37-48e3-93c4-ea20d8ae1cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:43.804042 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c236e1f9-8f37-48e3-93c4-ea20d8ae1cf0 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:43.805242 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c236e1f9-8f37-48e3-93c4-ea20d8ae1cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:43.869446 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1260/2539] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:43.877110 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb05ba-1b63-49e4-90eb-d247e8ab0b88 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:43.878739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb05ba-1b63-49e4-90eb-d247e8ab0b88 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:43.878739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb05ba-1b63-49e4-90eb-d247e8ab0b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:43.899885 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-57fb05ba-1b63-49e4-90eb-d247e8ab0b88 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:43.901087 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-57fb05ba-1b63-49e4-90eb-d247e8ab0b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:43.908907 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1280/2540] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:07:43.914246 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236e1f9-8f37-48e3-93c4-ea20d8ae1cf0 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:43.932741 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236e1f9-8f37-48e3-93c4-ea20d8ae1cf0 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:43.945188 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9578c9-3020-440f-863b-09f17a3202c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:43.945804 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9578c9-3020-440f-863b-09f17a3202c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:43.946082 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9578c9-3020-440f-863b-09f17a3202c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:43.946355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9578c9-3020-440f-863b-09f17a3202c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:43.947040 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9578c9-3020-440f-863b-09f17a3202c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:07:43.947338 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9578c9-3020-440f-863b-09f17a3202c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:43.948951 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9578c9-3020-440f-863b-09f17a3202c3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:43.964013 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1261/2541] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:07:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:07:43.968546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb05ba-1b63-49e4-90eb-d247e8ab0b88 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:43.979204 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb05ba-1b63-49e4-90eb-d247e8ab0b88 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:43.984342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04363bc-e8ba-4ec6-9cd5-82e286c6d01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:43.984948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04363bc-e8ba-4ec6-9cd5-82e286c6d01a admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:43.986152 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04363bc-e8ba-4ec6-9cd5-82e286c6d01a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:43.986152 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04363bc-e8ba-4ec6-9cd5-82e286c6d01a admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:43.986377 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04363bc-e8ba-4ec6-9cd5-82e286c6d01a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:43.988121 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04363bc-e8ba-4ec6-9cd5-82e286c6d01a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:43.998611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:44.097750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1281/2542] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:07:43 2022] POST /identity/v3/projects => generated 359 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:07:44.101834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9578c9-3020-440f-863b-09f17a3202c3 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:44.113601 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9578c9-3020-440f-863b-09f17a3202c3 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:44.126784 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:44.176876 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6b659-9eb3-4a16-858e-90b7167aa731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:44.177492 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6b659-9eb3-4a16-858e-90b7167aa731 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:44.178018 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6b659-9eb3-4a16-858e-90b7167aa731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:44.178018 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6b659-9eb3-4a16-858e-90b7167aa731 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:44.179357 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6b659-9eb3-4a16-858e-90b7167aa731 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:44.180783 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6b659-9eb3-4a16-858e-90b7167aa731 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:44.267957 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1262/2543] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:07:44 2022] POST /identity/v3/users => generated 280 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:07:44.273190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04363bc-e8ba-4ec6-9cd5-82e286c6d01a admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:44.283869 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04363bc-e8ba-4ec6-9cd5-82e286c6d01a admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:44.288885 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9b814-e451-426b-b3e3-408d55b2255b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:44.289462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9b814-e451-426b-b3e3-408d55b2255b admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:44.289731 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9b814-e451-426b-b3e3-408d55b2255b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:44.289995 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9b814-e451-426b-b3e3-408d55b2255b admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:44.290617 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e9b814-e451-426b-b3e3-408d55b2255b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:44.292216 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e9b814-e451-426b-b3e3-408d55b2255b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:44.302345 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1282/2544] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:07:44.307146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6b659-9eb3-4a16-858e-90b7167aa731 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:44.317141 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6b659-9eb3-4a16-858e-90b7167aa731 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:44.323130 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d51bc-8670-4cc4-bc90-33f9db38208b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:44.323714 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d51bc-8670-4cc4-bc90-33f9db38208b admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:44.323994 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d51bc-8670-4cc4-bc90-33f9db38208b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:44.324267 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d51bc-8670-4cc4-bc90-33f9db38208b admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:44.325166 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20d51bc-8670-4cc4-bc90-33f9db38208b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:44.326875 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20d51bc-8670-4cc4-bc90-33f9db38208b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:44.342619 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1263/2545] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:07:44.343766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9b814-e451-426b-b3e3-408d55b2255b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:44.354866 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9b814-e451-426b-b3e3-408d55b2255b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:44.360317 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1984d1-ea76-4efd-af62-098c321cd6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:44.360998 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1984d1-ea76-4efd-af62-098c321cd6af admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:44.361272 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1984d1-ea76-4efd-af62-098c321cd6af admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:44.361543 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1984d1-ea76-4efd-af62-098c321cd6af admin admin] PATH_INFO: `/v3/projects/79b016af9251499cba8af91a8ef93fad/users/81f28c1541b7404b9fc6b0250c40920c/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:44.378039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1984d1-ea76-4efd-af62-098c321cd6af admin admin] RBAC: Authorizing `identity:create_grant(project_id=79b016af9251499cba8af91a8ef93fad, user_id=81f28c1541b7404b9fc6b0250c40920c, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:44.381592 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1984d1-ea76-4efd-af62-098c321cd6af admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:44.479332 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1283/2546] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:07:44 2022] PUT /identity/v3/projects/79b016af9251499cba8af91a8ef93fad/users/81f28c1541b7404b9fc6b0250c40920c/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:44.546891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a58c33-020c-4788-9a37-303a565b4689 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:44.547085 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a58c33-020c-4788-9a37-303a565b4689 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:44.547354 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a58c33-020c-4788-9a37-303a565b4689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:44.578210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-71a58c33-020c-4788-9a37-303a565b4689 None None] MFA Rules not processed for user `81f28c1541b7404b9fc6b0250c40920c`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:44.579460 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-71a58c33-020c-4788-9a37-303a565b4689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:44.647641 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1264/2547] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:44 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:44.682526 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1984d1-ea76-4efd-af62-098c321cd6af admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:44.684371 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3c1984d1-ea76-4efd-af62-098c321cd6af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:44.748505 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1984d1-ea76-4efd-af62-098c321cd6af admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:44.752644 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc5c97-b3af-4a59-a584-bf42f9eb80ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:44.753240 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc5c97-b3af-4a59-a584-bf42f9eb80ae service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:44.753511 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc5c97-b3af-4a59-a584-bf42f9eb80ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:44.753775 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc5c97-b3af-4a59-a584-bf42f9eb80ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:44.756822 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df98ed-4c8b-42ea-85cd-5889ccad646a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:44.757112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df98ed-4c8b-42ea-85cd-5889ccad646a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:44.757384 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df98ed-4c8b-42ea-85cd-5889ccad646a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:44.762858 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc5c97-b3af-4a59-a584-bf42f9eb80ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:44.764303 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc5c97-b3af-4a59-a584-bf42f9eb80ae service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:44.773707 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1284/2548] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:07:44 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:07:44.776956 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f6df98ed-4c8b-42ea-85cd-5889ccad646a None None] MFA Rules not processed for user `b0a879085a194b0687e1cdcbafd48033`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:44.778299 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f6df98ed-4c8b-42ea-85cd-5889ccad646a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:44.791609 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc5c97-b3af-4a59-a584-bf42f9eb80ae service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:44.839057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1265/2549] 10.222.0.17 () {58 vars in 967 bytes} [Sun Sep 11 00:07:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:07:44.853288 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df98ed-4c8b-42ea-85cd-5889ccad646a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:44.854731 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f6df98ed-4c8b-42ea-85cd-5889ccad646a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:44.855503 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc5c97-b3af-4a59-a584-bf42f9eb80ae service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:44.860140 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6a737-9a70-4202-abe8-e031f3d3608d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:44.860769 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6a737-9a70-4202-abe8-e031f3d3608d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:44.861035 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6a737-9a70-4202-abe8-e031f3d3608d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:44.861303 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6a737-9a70-4202-abe8-e031f3d3608d admin admin] PATH_INFO: `/v3/users/81f28c1541b7404b9fc6b0250c40920c` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:44.862413 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6a737-9a70-4202-abe8-e031f3d3608d admin admin] RBAC: Authorizing `identity:delete_user(user_id=81f28c1541b7404b9fc6b0250c40920c)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:44.864337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6a737-9a70-4202-abe8-e031f3d3608d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:44.885924 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:44.924231 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df98ed-4c8b-42ea-85cd-5889ccad646a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:44.929706 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d13fed-7e9e-403f-bafc-cb0d6b3388bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '564e65585e9543e286b6041af870cf41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:44.935258 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d13fed-7e9e-403f-bafc-cb0d6b3388bc service swift] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:44.935258 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d13fed-7e9e-403f-bafc-cb0d6b3388bc service swift] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:44.935258 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d13fed-7e9e-403f-bafc-cb0d6b3388bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:44.938803 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d13fed-7e9e-403f-bafc-cb0d6b3388bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:44.940457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d13fed-7e9e-403f-bafc-cb0d6b3388bc service swift] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:44.948912 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1266/2550] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Sep 11 00:07:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 00:07:45.095091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e4c6a737-9a70-4202-abe8-e031f3d3608d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81f28c1541b7404b9fc6b0250c40920c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:45.104513 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:45.109534 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e4c6a737-9a70-4202-abe8-e031f3d3608d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81f28c1541b7404b9fc6b0250c40920c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:45.117652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e4c6a737-9a70-4202-abe8-e031f3d3608d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81f28c1541b7404b9fc6b0250c40920c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:45.118379 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e4c6a737-9a70-4202-abe8-e031f3d3608d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81f28c1541b7404b9fc6b0250c40920c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:45.182466 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e4c6a737-9a70-4202-abe8-e031f3d3608d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81f28c1541b7404b9fc6b0250c40920c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:45.197960 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1285/2551] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:07:44 2022] DELETE /identity/v3/users/81f28c1541b7404b9fc6b0250c40920c => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:45.202332 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62191643-74f2-44d8-9960-152ca57f4c65 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:45.202615 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62191643-74f2-44d8-9960-152ca57f4c65 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:45.202952 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62191643-74f2-44d8-9960-152ca57f4c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:45.223868 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-62191643-74f2-44d8-9960-152ca57f4c65 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:45.225090 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-62191643-74f2-44d8-9960-152ca57f4c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:45.299332 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1267/2552] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:45.310238 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6a737-9a70-4202-abe8-e031f3d3608d admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:45.311178 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e4c6a737-9a70-4202-abe8-e031f3d3608d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:45.371515 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6a737-9a70-4202-abe8-e031f3d3608d admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:45.375182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e1dbcc-e799-4f61-a2c1-69752ccd61b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:45.375774 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1dbcc-e799-4f61-a2c1-69752ccd61b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:45.376045 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1dbcc-e799-4f61-a2c1-69752ccd61b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:45.376333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1dbcc-e799-4f61-a2c1-69752ccd61b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:45.385888 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e1dbcc-e799-4f61-a2c1-69752ccd61b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:45.387356 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e1dbcc-e799-4f61-a2c1-69752ccd61b4 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:45.397056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1286/2553] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:07:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:07:45.801040 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62191643-74f2-44d8-9960-152ca57f4c65 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:45.802840 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-62191643-74f2-44d8-9960-152ca57f4c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:45.816466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62191643-74f2-44d8-9960-152ca57f4c65 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:45.823965 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09cb0dd-d3af-4972-b634-6ef9b3960f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:45.824872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09cb0dd-d3af-4972-b634-6ef9b3960f73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:45.825150 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09cb0dd-d3af-4972-b634-6ef9b3960f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:45.831559 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09cb0dd-d3af-4972-b634-6ef9b3960f73 admin admin] PATH_INFO: `/v3/projects/79b016af9251499cba8af91a8ef93fad` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:45.831559 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09cb0dd-d3af-4972-b634-6ef9b3960f73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79b016af9251499cba8af91a8ef93fad)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:45.831559 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09cb0dd-d3af-4972-b634-6ef9b3960f73 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:45.931156 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d09cb0dd-d3af-4972-b634-6ef9b3960f73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79b016af9251499cba8af91a8ef93fad', 'request_id': 'req-d09cb0dd-d3af-4972-b634-6ef9b3960f73'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:45.937375 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d09cb0dd-d3af-4972-b634-6ef9b3960f73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79b016af9251499cba8af91a8ef93fad', 'request_id': 'req-d09cb0dd-d3af-4972-b634-6ef9b3960f73'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:45.944722 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d09cb0dd-d3af-4972-b634-6ef9b3960f73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79b016af9251499cba8af91a8ef93fad', 'request_id': 'req-d09cb0dd-d3af-4972-b634-6ef9b3960f73'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:46.027948 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d09cb0dd-d3af-4972-b634-6ef9b3960f73 admin admin] The token cache is being invalidate because project 79b016af9251499cba8af91a8ef93fad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:07:46.028262 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d09cb0dd-d3af-4972-b634-6ef9b3960f73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:46.028879 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d09cb0dd-d3af-4972-b634-6ef9b3960f73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:07:46.030391 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1268/2554] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:07:45 2022] DELETE /identity/v3/projects/79b016af9251499cba8af91a8ef93fad => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:46.035667 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e830835-ebb7-49bc-8946-cfa0904798cc None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:46.036146 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e830835-ebb7-49bc-8946-cfa0904798cc None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:46.036257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e830835-ebb7-49bc-8946-cfa0904798cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:46.056210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-5e830835-ebb7-49bc-8946-cfa0904798cc None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:46.057407 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5e830835-ebb7-49bc-8946-cfa0904798cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:46.119100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1287/2555] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:07:46.126782 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792df8d-6ef9-40aa-b021-4515c6f91353 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:46.127066 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792df8d-6ef9-40aa-b021-4515c6f91353 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:46.127339 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792df8d-6ef9-40aa-b021-4515c6f91353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:46.147373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-4792df8d-6ef9-40aa-b021-4515c6f91353 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:46.148576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4792df8d-6ef9-40aa-b021-4515c6f91353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:46.156440 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1269/2556] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:46.160779 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e830835-ebb7-49bc-8946-cfa0904798cc None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:46.170986 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e830835-ebb7-49bc-8946-cfa0904798cc None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:46.175447 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47758cd-aecb-4c29-b4b1-c7f89fda58fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:46.176032 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47758cd-aecb-4c29-b4b1-c7f89fda58fb admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:46.176306 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47758cd-aecb-4c29-b4b1-c7f89fda58fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:46.176572 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47758cd-aecb-4c29-b4b1-c7f89fda58fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:46.177199 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47758cd-aecb-4c29-b4b1-c7f89fda58fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:07:46.178143 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47758cd-aecb-4c29-b4b1-c7f89fda58fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:46.179774 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47758cd-aecb-4c29-b4b1-c7f89fda58fb admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:46.191166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1288/2557] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:07:46.195665 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792df8d-6ef9-40aa-b021-4515c6f91353 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:46.205276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792df8d-6ef9-40aa-b021-4515c6f91353 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:46.210245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0edd5c4-7367-4859-89e2-178e9f6072a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:46.210876 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0edd5c4-7367-4859-89e2-178e9f6072a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:46.211155 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0edd5c4-7367-4859-89e2-178e9f6072a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:46.211426 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0edd5c4-7367-4859-89e2-178e9f6072a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:46.212125 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0edd5c4-7367-4859-89e2-178e9f6072a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:46.214027 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0edd5c4-7367-4859-89e2-178e9f6072a4 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:46.292940 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1270/2558] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:07:46 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:07:46.299700 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47758cd-aecb-4c29-b4b1-c7f89fda58fb admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:46.310873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47758cd-aecb-4c29-b4b1-c7f89fda58fb admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:46.371324 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ac72d-6f0c-4536-bcc4-0cbd68499299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:46.371922 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ac72d-6f0c-4536-bcc4-0cbd68499299 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:46.372199 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ac72d-6f0c-4536-bcc4-0cbd68499299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:46.372467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ac72d-6f0c-4536-bcc4-0cbd68499299 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:46.373090 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ac72d-6f0c-4536-bcc4-0cbd68499299 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:46.375044 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ac72d-6f0c-4536-bcc4-0cbd68499299 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:46.386608 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:46.433962 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1289/2559] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:07:46 2022] POST /identity/v3/users => generated 286 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:07:46.437940 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0edd5c4-7367-4859-89e2-178e9f6072a4 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:46.448828 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0edd5c4-7367-4859-89e2-178e9f6072a4 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:46.453779 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83863fe8-fb8e-4a92-8979-704403d2c9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:46.454364 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83863fe8-fb8e-4a92-8979-704403d2c9fa admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:46.454642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83863fe8-fb8e-4a92-8979-704403d2c9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:46.454951 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83863fe8-fb8e-4a92-8979-704403d2c9fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:46.455585 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83863fe8-fb8e-4a92-8979-704403d2c9fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:46.457182 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83863fe8-fb8e-4a92-8979-704403d2c9fa admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:46.463173 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:46.468130 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1271/2560] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:07:46.472515 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ac72d-6f0c-4536-bcc4-0cbd68499299 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:46.483100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ac72d-6f0c-4536-bcc4-0cbd68499299 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:46.487711 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7693a9-e00e-4212-82b8-6a8fbd311fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:46.488300 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7693a9-e00e-4212-82b8-6a8fbd311fe3 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:46.488599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7693a9-e00e-4212-82b8-6a8fbd311fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:46.488871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7693a9-e00e-4212-82b8-6a8fbd311fe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:46.489492 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7693a9-e00e-4212-82b8-6a8fbd311fe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:46.491217 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7693a9-e00e-4212-82b8-6a8fbd311fe3 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:46.501601 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1290/2561] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:07:46.506556 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83863fe8-fb8e-4a92-8979-704403d2c9fa admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:46.516718 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83863fe8-fb8e-4a92-8979-704403d2c9fa admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:46.526430 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080b623-5cbe-42d1-ab99-12a3faf60861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:46.527113 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080b623-5cbe-42d1-ab99-12a3faf60861 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:46.527390 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080b623-5cbe-42d1-ab99-12a3faf60861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:46.527658 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080b623-5cbe-42d1-ab99-12a3faf60861 admin admin] PATH_INFO: `/v3/projects/49b0d0136b404c68902274c0e571fb3a/users/32b5ea2215fd4c38aabaf375db991d06/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:46.542147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0080b623-5cbe-42d1-ab99-12a3faf60861 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49b0d0136b404c68902274c0e571fb3a, user_id=32b5ea2215fd4c38aabaf375db991d06, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:46.544842 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0080b623-5cbe-42d1-ab99-12a3faf60861 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:46.598412 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1272/2562] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:07:46 2022] PUT /identity/v3/projects/49b0d0136b404c68902274c0e571fb3a/users/32b5ea2215fd4c38aabaf375db991d06/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:07:46.603885 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633082b-575c-441d-ae4e-427219374c9a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:46.604182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633082b-575c-441d-ae4e-427219374c9a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:46.604453 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633082b-575c-441d-ae4e-427219374c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:46.623574 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b633082b-575c-441d-ae4e-427219374c9a None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:46.624774 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b633082b-575c-441d-ae4e-427219374c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:46.684308 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1291/2563] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:46.694584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080b623-5cbe-42d1-ab99-12a3faf60861 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:46.696289 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0080b623-5cbe-42d1-ab99-12a3faf60861 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:46.752574 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080b623-5cbe-42d1-ab99-12a3faf60861 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:46.756916 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0dc4d6-3ed7-4d89-a421-e77986073448 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:46.757503 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0dc4d6-3ed7-4d89-a421-e77986073448 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:46.757782 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0dc4d6-3ed7-4d89-a421-e77986073448 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:46.758048 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0dc4d6-3ed7-4d89-a421-e77986073448 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:46.766703 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0dc4d6-3ed7-4d89-a421-e77986073448 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:46.768132 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0dc4d6-3ed7-4d89-a421-e77986073448 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:46.777325 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1273/2564] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:07:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:07:52.389573 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497d4c2-f4ea-48f3-9617-b06a78269c25 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:52.390040 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497d4c2-f4ea-48f3-9617-b06a78269c25 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:52.390150 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497d4c2-f4ea-48f3-9617-b06a78269c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:52.431600 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b497d4c2-f4ea-48f3-9617-b06a78269c25 None None] MFA Rules not processed for user `32b5ea2215fd4c38aabaf375db991d06`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:52.433037 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b497d4c2-f4ea-48f3-9617-b06a78269c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:52.511086 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1292/2565] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:52.515822 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0dc4d6-3ed7-4d89-a421-e77986073448 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:52.526782 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0dc4d6-3ed7-4d89-a421-e77986073448 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:52.532104 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9a203-7fc5-4e8f-9bc6-e12c845aacf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:52.532700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9a203-7fc5-4e8f-9bc6-e12c845aacf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:52.532972 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9a203-7fc5-4e8f-9bc6-e12c845aacf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:52.533241 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9a203-7fc5-4e8f-9bc6-e12c845aacf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:52.533864 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f9a203-7fc5-4e8f-9bc6-e12c845aacf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:52.535828 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f9a203-7fc5-4e8f-9bc6-e12c845aacf7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:52.640814 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1274/2566] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:07:52 2022] POST /identity/v3/projects => generated 369 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:07:52.644961 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497d4c2-f4ea-48f3-9617-b06a78269c25 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:52.655779 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497d4c2-f4ea-48f3-9617-b06a78269c25 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:52.720528 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05295389-849a-456e-a2b2-08bdb9b3c771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:52.721154 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05295389-849a-456e-a2b2-08bdb9b3c771 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:52.721426 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05295389-849a-456e-a2b2-08bdb9b3c771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:52.721688 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05295389-849a-456e-a2b2-08bdb9b3c771 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:52.722310 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05295389-849a-456e-a2b2-08bdb9b3c771 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:52.724232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05295389-849a-456e-a2b2-08bdb9b3c771 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:52.794014 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1293/2567] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:07:52 2022] POST /identity/v3/users => generated 291 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:07:52.798244 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9a203-7fc5-4e8f-9bc6-e12c845aacf7 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:52.809685 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9a203-7fc5-4e8f-9bc6-e12c845aacf7 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:52.815881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ca4d1-601b-4453-8acf-4f2345b513f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:52.818987 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ca4d1-601b-4453-8acf-4f2345b513f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:52.819258 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ca4d1-601b-4453-8acf-4f2345b513f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:52.819628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ca4d1-601b-4453-8acf-4f2345b513f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:52.820284 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ca4d1-601b-4453-8acf-4f2345b513f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:52.821953 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ca4d1-601b-4453-8acf-4f2345b513f0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:52.834063 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1275/2568] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:07:52.838158 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05295389-849a-456e-a2b2-08bdb9b3c771 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:52.844994 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:52.848964 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05295389-849a-456e-a2b2-08bdb9b3c771 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:52.853749 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43f5fd-005f-4abe-87bf-721d54986d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:52.854399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43f5fd-005f-4abe-87bf-721d54986d49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:52.854698 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43f5fd-005f-4abe-87bf-721d54986d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:52.854974 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43f5fd-005f-4abe-87bf-721d54986d49 admin admin] PATH_INFO: `/v3/projects/86899cff62834c7fb481b7c87ca10dc6/users/3c3b85078428455da0c5be75378b5cd7/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:52.877216 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43f5fd-005f-4abe-87bf-721d54986d49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86899cff62834c7fb481b7c87ca10dc6, user_id=3c3b85078428455da0c5be75378b5cd7, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:52.879896 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43f5fd-005f-4abe-87bf-721d54986d49 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:52.954171 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1294/2569] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:07:52 2022] PUT /identity/v3/projects/86899cff62834c7fb481b7c87ca10dc6/users/3c3b85078428455da0c5be75378b5cd7/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:07:52.960017 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ca4d1-601b-4453-8acf-4f2345b513f0 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:52.972201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ca4d1-601b-4453-8acf-4f2345b513f0 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:53.040294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e155d-2780-45df-b4e7-3f2b25441434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:53.040966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e155d-2780-45df-b4e7-3f2b25441434 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:53.041847 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e155d-2780-45df-b4e7-3f2b25441434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:53.041847 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e155d-2780-45df-b4e7-3f2b25441434 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:53.042797 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e155d-2780-45df-b4e7-3f2b25441434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:53.049265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e155d-2780-45df-b4e7-3f2b25441434 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:53.057958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1276/2570] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:07:53.062138 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43f5fd-005f-4abe-87bf-721d54986d49 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:53.084336 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43f5fd-005f-4abe-87bf-721d54986d49 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:53.092317 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213ff76-0b65-4179-8967-6f01c6aa3ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:53.094749 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213ff76-0b65-4179-8967-6f01c6aa3ec3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:53.094749 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213ff76-0b65-4179-8967-6f01c6aa3ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:53.094749 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213ff76-0b65-4179-8967-6f01c6aa3ec3 admin admin] PATH_INFO: `/v3/projects/86899cff62834c7fb481b7c87ca10dc6/users/3c3b85078428455da0c5be75378b5cd7/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:53.096806 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213ff76-0b65-4179-8967-6f01c6aa3ec3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86899cff62834c7fb481b7c87ca10dc6, user_id=3c3b85078428455da0c5be75378b5cd7, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:53.100187 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213ff76-0b65-4179-8967-6f01c6aa3ec3 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:53.111658 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1295/2571] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:07:53 2022] PUT /identity/v3/projects/86899cff62834c7fb481b7c87ca10dc6/users/3c3b85078428455da0c5be75378b5cd7/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:07:58.984801 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ba959-14d9-4fcf-9f4a-e05ec65a8c9d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:58.985276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ba959-14d9-4fcf-9f4a-e05ec65a8c9d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:58.985276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ba959-14d9-4fcf-9f4a-e05ec65a8c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:59.028717 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-af0ba959-14d9-4fcf-9f4a-e05ec65a8c9d None None] MFA Rules not processed for user `3c3b85078428455da0c5be75378b5cd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:07:59.029378 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-af0ba959-14d9-4fcf-9f4a-e05ec65a8c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:59.081142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:07:59.150410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1277/2572] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:07:58 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:07:59.165346 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213ff76-0b65-4179-8967-6f01c6aa3ec3 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:07:59.167317 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9213ff76-0b65-4179-8967-6f01c6aa3ec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:07:59.258582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213ff76-0b65-4179-8967-6f01c6aa3ec3 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:07:59.264510 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92bc78f-5cff-4e4e-ba67-419fb3ca9217 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:07:59.265798 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92bc78f-5cff-4e4e-ba67-419fb3ca9217 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:07:59.266232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92bc78f-5cff-4e4e-ba67-419fb3ca9217 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:07:59.266653 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92bc78f-5cff-4e4e-ba67-419fb3ca9217 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:07:59.345676 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92bc78f-5cff-4e4e-ba67-419fb3ca9217 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:07:59.347253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92bc78f-5cff-4e4e-ba67-419fb3ca9217 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:07:59.357292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1296/2573] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:07:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:08:01.943893 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f2680-4c06-4d7b-8a74-be7c67ff926d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:01.944529 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f2680-4c06-4d7b-8a74-be7c67ff926d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:01.944529 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f2680-4c06-4d7b-8a74-be7c67ff926d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:01.967215 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-e17f2680-4c06-4d7b-8a74-be7c67ff926d None None] MFA Rules not processed for user `32b5ea2215fd4c38aabaf375db991d06`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:01.968615 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e17f2680-4c06-4d7b-8a74-be7c67ff926d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:01.976700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1278/2574] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:08:08.388019 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92bc78f-5cff-4e4e-ba67-419fb3ca9217 service nova] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:08.389423 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c92bc78f-5cff-4e4e-ba67-419fb3ca9217 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:08.451249 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92bc78f-5cff-4e4e-ba67-419fb3ca9217 service nova] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:08.455023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42221e-b92a-4268-ab65-a4de52e42aaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:08.455632 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42221e-b92a-4268-ab65-a4de52e42aaf service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:08.455909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42221e-b92a-4268-ab65-a4de52e42aaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:08.456182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42221e-b92a-4268-ab65-a4de52e42aaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:08.457886 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ff42221e-b92a-4268-ab65-a4de52e42aaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:08.476708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:08.524460 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff42221e-b92a-4268-ab65-a4de52e42aaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:08.526180 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff42221e-b92a-4268-ab65-a4de52e42aaf service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:08.537441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1297/2575] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:08:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:08:11.963371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=18 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:12.326866 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f2680-4c06-4d7b-8a74-be7c67ff926d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:12.328491 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e17f2680-4c06-4d7b-8a74-be7c67ff926d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:12.410600 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f2680-4c06-4d7b-8a74-be7c67ff926d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:12.422874 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8aa0f8-3d08-47a5-9a07-0328404c50dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cb1d85a338e40f18e61887a1becb906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:12.422874 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8aa0f8-3d08-47a5-9a07-0328404c50dd service placement] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:12.422874 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8aa0f8-3d08-47a5-9a07-0328404c50dd service placement] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:12.422874 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8aa0f8-3d08-47a5-9a07-0328404c50dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:12.423472 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9a8aa0f8-3d08-47a5-9a07-0328404c50dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:12.432719 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8aa0f8-3d08-47a5-9a07-0328404c50dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:12.434365 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8aa0f8-3d08-47a5-9a07-0328404c50dd service placement] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:12.445684 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1279/2576] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:08:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:08:12.455965 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42221e-b92a-4268-ab65-a4de52e42aaf service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:12.467268 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ff42221e-b92a-4268-ab65-a4de52e42aaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:12.478552 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42221e-b92a-4268-ab65-a4de52e42aaf service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:12.483525 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a9320-a62b-4d2f-b106-ffd99bc6a058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:12.484229 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a9320-a62b-4d2f-b106-ffd99bc6a058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:12.484583 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a9320-a62b-4d2f-b106-ffd99bc6a058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:12.484916 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a9320-a62b-4d2f-b106-ffd99bc6a058 admin admin] PATH_INFO: `/v3/users/9950f578162449c0845c3aa4aab692d4` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:12.486109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a9320-a62b-4d2f-b106-ffd99bc6a058 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9950f578162449c0845c3aa4aab692d4)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:12.488142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a9320-a62b-4d2f-b106-ffd99bc6a058 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:12.665205 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2a8a9320-a62b-4d2f-b106-ffd99bc6a058 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9950f578162449c0845c3aa4aab692d4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:12.676893 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2a8a9320-a62b-4d2f-b106-ffd99bc6a058 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9950f578162449c0845c3aa4aab692d4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:12.684730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2a8a9320-a62b-4d2f-b106-ffd99bc6a058 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9950f578162449c0845c3aa4aab692d4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:12.685394 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2a8a9320-a62b-4d2f-b106-ffd99bc6a058 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9950f578162449c0845c3aa4aab692d4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:12.770537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2a8a9320-a62b-4d2f-b106-ffd99bc6a058 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9950f578162449c0845c3aa4aab692d4'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:12.777948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1298/2577] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:08:12 2022] DELETE /identity/v3/users/9950f578162449c0845c3aa4aab692d4 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:13.023534 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8aa0f8-3d08-47a5-9a07-0328404c50dd service placement] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:13.034339 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8aa0f8-3d08-47a5-9a07-0328404c50dd service placement] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:13.097948 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b333bd2-f36b-4503-895f-bc6797e29ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:13.098572 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b333bd2-f36b-4503-895f-bc6797e29ffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:13.098878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b333bd2-f36b-4503-895f-bc6797e29ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:13.099188 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b333bd2-f36b-4503-895f-bc6797e29ffd admin admin] PATH_INFO: `/v3/projects/65f0209a8e48496c8d88c885f8364cbd` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:13.100312 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b333bd2-f36b-4503-895f-bc6797e29ffd admin admin] RBAC: Authorizing `identity:delete_project(project_id=65f0209a8e48496c8d88c885f8364cbd)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:13.102210 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b333bd2-f36b-4503-895f-bc6797e29ffd admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:13.195562 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2b333bd2-f36b-4503-895f-bc6797e29ffd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65f0209a8e48496c8d88c885f8364cbd', 'request_id': 'req-2b333bd2-f36b-4503-895f-bc6797e29ffd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:13.202069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2b333bd2-f36b-4503-895f-bc6797e29ffd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65f0209a8e48496c8d88c885f8364cbd', 'request_id': 'req-2b333bd2-f36b-4503-895f-bc6797e29ffd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:13.207595 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2b333bd2-f36b-4503-895f-bc6797e29ffd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65f0209a8e48496c8d88c885f8364cbd', 'request_id': 'req-2b333bd2-f36b-4503-895f-bc6797e29ffd'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:13.255038 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2b333bd2-f36b-4503-895f-bc6797e29ffd admin admin] The token cache is being invalidate because project 65f0209a8e48496c8d88c885f8364cbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:08:13.255450 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2b333bd2-f36b-4503-895f-bc6797e29ffd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:13.256224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2b333bd2-f36b-4503-895f-bc6797e29ffd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:13.257762 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1280/2578] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:08:13 2022] DELETE /identity/v3/projects/65f0209a8e48496c8d88c885f8364cbd => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:13.262609 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741c426-1797-4e47-9d7b-7666b42cce21 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:13.262915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741c426-1797-4e47-9d7b-7666b42cce21 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:13.263193 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741c426-1797-4e47-9d7b-7666b42cce21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:13.284197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-c741c426-1797-4e47-9d7b-7666b42cce21 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:13.285389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c741c426-1797-4e47-9d7b-7666b42cce21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:13.348407 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1299/2579] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:08:13.357935 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2ef2d-18a3-4a61-b862-8c029dc9ee5d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:13.358224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2ef2d-18a3-4a61-b862-8c029dc9ee5d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:13.358512 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2ef2d-18a3-4a61-b862-8c029dc9ee5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:13.379965 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-74c2ef2d-18a3-4a61-b862-8c029dc9ee5d None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:13.381444 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-74c2ef2d-18a3-4a61-b862-8c029dc9ee5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:13.389405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1281/2580] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:08:13.393714 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741c426-1797-4e47-9d7b-7666b42cce21 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:13.403690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741c426-1797-4e47-9d7b-7666b42cce21 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:13.408322 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099b3dd-aad6-4a55-98e7-08f33b63bc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:13.408943 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099b3dd-aad6-4a55-98e7-08f33b63bc0b admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:13.409216 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099b3dd-aad6-4a55-98e7-08f33b63bc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:13.409494 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099b3dd-aad6-4a55-98e7-08f33b63bc0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:13.410130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9099b3dd-aad6-4a55-98e7-08f33b63bc0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:08:13.410435 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9099b3dd-aad6-4a55-98e7-08f33b63bc0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:13.412566 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9099b3dd-aad6-4a55-98e7-08f33b63bc0b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:13.424859 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1300/2581] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:08:13.429699 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2ef2d-18a3-4a61-b862-8c029dc9ee5d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:13.438876 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2ef2d-18a3-4a61-b862-8c029dc9ee5d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:13.444497 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba07b14-1935-46c8-be63-0b86bd608c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:13.445100 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba07b14-1935-46c8-be63-0b86bd608c59 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:13.445380 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba07b14-1935-46c8-be63-0b86bd608c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:13.445651 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba07b14-1935-46c8-be63-0b86bd608c59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:13.446279 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba07b14-1935-46c8-be63-0b86bd608c59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:13.448231 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba07b14-1935-46c8-be63-0b86bd608c59 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:13.461336 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:13.519889 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1282/2582] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:08:13 2022] POST /identity/v3/projects => generated 393 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:08:13.524807 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099b3dd-aad6-4a55-98e7-08f33b63bc0b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:13.535294 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099b3dd-aad6-4a55-98e7-08f33b63bc0b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:13.546055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:13.601038 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515bfa1-59de-40de-8204-b9c9dafeee9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:13.601656 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515bfa1-59de-40de-8204-b9c9dafeee9a admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:13.601929 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515bfa1-59de-40de-8204-b9c9dafeee9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:13.602221 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515bfa1-59de-40de-8204-b9c9dafeee9a admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:13.602955 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6515bfa1-59de-40de-8204-b9c9dafeee9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:13.604895 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6515bfa1-59de-40de-8204-b9c9dafeee9a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:13.708281 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1301/2583] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:08:13 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:08:13.713012 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba07b14-1935-46c8-be63-0b86bd608c59 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:13.723188 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba07b14-1935-46c8-be63-0b86bd608c59 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:13.728247 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbc930-504b-442c-a81a-631da7b836c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:13.728867 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbc930-504b-442c-a81a-631da7b836c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:13.729135 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbc930-504b-442c-a81a-631da7b836c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:13.729406 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbc930-504b-442c-a81a-631da7b836c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:13.730035 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbbc930-504b-442c-a81a-631da7b836c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:13.731659 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbbc930-504b-442c-a81a-631da7b836c4 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:13.742968 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1283/2584] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:08:13.747107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515bfa1-59de-40de-8204-b9c9dafeee9a admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:13.758015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515bfa1-59de-40de-8204-b9c9dafeee9a admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:13.762647 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6bec71-682b-4073-b8da-35a948180176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:13.763263 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6bec71-682b-4073-b8da-35a948180176 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:13.763542 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6bec71-682b-4073-b8da-35a948180176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:13.763812 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6bec71-682b-4073-b8da-35a948180176 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:13.764446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6bec71-682b-4073-b8da-35a948180176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:13.766029 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6bec71-682b-4073-b8da-35a948180176 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:13.776388 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1302/2585] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:08:13.780522 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbc930-504b-442c-a81a-631da7b836c4 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:13.790572 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbc930-504b-442c-a81a-631da7b836c4 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:13.795473 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e6036-c5c2-4b06-8954-79ef2561c425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:13.796146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e6036-c5c2-4b06-8954-79ef2561c425 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:13.796417 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e6036-c5c2-4b06-8954-79ef2561c425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:13.796692 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e6036-c5c2-4b06-8954-79ef2561c425 admin admin] PATH_INFO: `/v3/projects/09c24bf2f86b44728f6e5fdc959665ab/users/010035bf48314776b8909bbd7eac8fd3/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:13.812041 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e6036-c5c2-4b06-8954-79ef2561c425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09c24bf2f86b44728f6e5fdc959665ab, user_id=010035bf48314776b8909bbd7eac8fd3, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:13.814712 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e6036-c5c2-4b06-8954-79ef2561c425 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:13.885651 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1284/2586] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:08:13 2022] PUT /identity/v3/projects/09c24bf2f86b44728f6e5fdc959665ab/users/010035bf48314776b8909bbd7eac8fd3/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:08:13.895996 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434a886-5366-4852-918a-7bfd7cdd8f9c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:13.896273 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434a886-5366-4852-918a-7bfd7cdd8f9c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:13.896543 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434a886-5366-4852-918a-7bfd7cdd8f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:13.924547 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-6434a886-5366-4852-918a-7bfd7cdd8f9c None None] MFA Rules not processed for user `010035bf48314776b8909bbd7eac8fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:13.925731 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6434a886-5366-4852-918a-7bfd7cdd8f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:13.992365 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1303/2587] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:08:13.997015 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e6036-c5c2-4b06-8954-79ef2561c425 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:14.067213 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e6036-c5c2-4b06-8954-79ef2561c425 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:14.072871 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c2320-8b15-42d5-aa5a-f9deb4a37420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:14.073303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c2320-8b15-42d5-aa5a-f9deb4a37420 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:14.073565 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c2320-8b15-42d5-aa5a-f9deb4a37420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:14.073834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c2320-8b15-42d5-aa5a-f9deb4a37420 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:14.074473 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c2320-8b15-42d5-aa5a-f9deb4a37420 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:14.076750 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c2320-8b15-42d5-aa5a-f9deb4a37420 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:14.160121 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1285/2588] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:08:13 2022] POST /identity/v3/projects => generated 387 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:08:14.164522 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434a886-5366-4852-918a-7bfd7cdd8f9c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:14.175657 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434a886-5366-4852-918a-7bfd7cdd8f9c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:14.234574 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b315c80-f3b4-4db6-ba64-640ae036fc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:14.235211 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b315c80-f3b4-4db6-ba64-640ae036fc35 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:14.235482 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b315c80-f3b4-4db6-ba64-640ae036fc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:14.235750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b315c80-f3b4-4db6-ba64-640ae036fc35 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:14.236378 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b315c80-f3b4-4db6-ba64-640ae036fc35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:14.238256 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b315c80-f3b4-4db6-ba64-640ae036fc35 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:14.348062 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1304/2589] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:08:14 2022] POST /identity/v3/users => generated 300 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:08:14.352622 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c2320-8b15-42d5-aa5a-f9deb4a37420 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:14.363110 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c2320-8b15-42d5-aa5a-f9deb4a37420 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:14.368698 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14302a0-a65a-4caa-86d7-58ce4a043f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:14.369301 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14302a0-a65a-4caa-86d7-58ce4a043f67 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:14.369565 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14302a0-a65a-4caa-86d7-58ce4a043f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:14.369829 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14302a0-a65a-4caa-86d7-58ce4a043f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:14.370466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14302a0-a65a-4caa-86d7-58ce4a043f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:14.372299 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14302a0-a65a-4caa-86d7-58ce4a043f67 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:14.384015 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1286/2590] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:08:14.388330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b315c80-f3b4-4db6-ba64-640ae036fc35 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:14.398806 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b315c80-f3b4-4db6-ba64-640ae036fc35 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:14.403391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eda9e0-b2e9-4dab-a1eb-9614bba30c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:14.404027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eda9e0-b2e9-4dab-a1eb-9614bba30c0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:14.404306 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eda9e0-b2e9-4dab-a1eb-9614bba30c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:14.404571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eda9e0-b2e9-4dab-a1eb-9614bba30c0f admin admin] PATH_INFO: `/v3/projects/810da9939ba8433a8e426db6d7883de7/users/92e7bb3389c14a159c5450e3d769b123/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:14.412981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:14.419396 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eda9e0-b2e9-4dab-a1eb-9614bba30c0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=810da9939ba8433a8e426db6d7883de7, user_id=92e7bb3389c14a159c5450e3d769b123, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:14.422064 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eda9e0-b2e9-4dab-a1eb-9614bba30c0f admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:14.513775 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1305/2591] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:08:14 2022] PUT /identity/v3/projects/810da9939ba8433a8e426db6d7883de7/users/92e7bb3389c14a159c5450e3d769b123/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:14.517828 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14302a0-a65a-4caa-86d7-58ce4a043f67 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:14.528903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14302a0-a65a-4caa-86d7-58ce4a043f67 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:14.587287 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ad575-506a-4a2a-b67f-f3a97fa8604d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:14.587894 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ad575-506a-4a2a-b67f-f3a97fa8604d admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:14.588171 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ad575-506a-4a2a-b67f-f3a97fa8604d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:14.588444 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ad575-506a-4a2a-b67f-f3a97fa8604d admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:14.589079 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7ad575-506a-4a2a-b67f-f3a97fa8604d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:14.590707 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7ad575-506a-4a2a-b67f-f3a97fa8604d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:14.600496 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1287/2592] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:08:14.605354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eda9e0-b2e9-4dab-a1eb-9614bba30c0f admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:14.615771 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eda9e0-b2e9-4dab-a1eb-9614bba30c0f admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:14.620458 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2daf4-426f-4b61-99a9-7f9f40f47525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:14.621093 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2daf4-426f-4b61-99a9-7f9f40f47525 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:14.621364 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2daf4-426f-4b61-99a9-7f9f40f47525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:14.621634 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2daf4-426f-4b61-99a9-7f9f40f47525 admin admin] PATH_INFO: `/v3/projects/810da9939ba8433a8e426db6d7883de7/users/92e7bb3389c14a159c5450e3d769b123/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:14.623544 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2daf4-426f-4b61-99a9-7f9f40f47525 admin admin] RBAC: Authorizing `identity:create_grant(project_id=810da9939ba8433a8e426db6d7883de7, user_id=92e7bb3389c14a159c5450e3d769b123, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:14.626138 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2daf4-426f-4b61-99a9-7f9f40f47525 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:14.636566 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1306/2593] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:08:14 2022] PUT /identity/v3/projects/810da9939ba8433a8e426db6d7883de7/users/92e7bb3389c14a159c5450e3d769b123/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:14.646612 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f084ef8-4e3d-4c00-a4cf-0c64165b89cb None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:14.646927 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f084ef8-4e3d-4c00-a4cf-0c64165b89cb None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:14.647218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f084ef8-4e3d-4c00-a4cf-0c64165b89cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:14.674906 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5f084ef8-4e3d-4c00-a4cf-0c64165b89cb None None] MFA Rules not processed for user `92e7bb3389c14a159c5450e3d769b123`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:14.676112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5f084ef8-4e3d-4c00-a4cf-0c64165b89cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:14.688961 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:14.745274 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1288/2594] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:14 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:08:14.755790 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2daf4-426f-4b61-99a9-7f9f40f47525 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:14.758393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-37b2daf4-426f-4b61-99a9-7f9f40f47525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:14.816238 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2daf4-426f-4b61-99a9-7f9f40f47525 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:14.820824 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317098bb-887a-4ba3-88ef-2cd6ad96018b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:14.821647 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317098bb-887a-4ba3-88ef-2cd6ad96018b service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:14.821918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317098bb-887a-4ba3-88ef-2cd6ad96018b service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:14.822185 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317098bb-887a-4ba3-88ef-2cd6ad96018b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:14.831914 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317098bb-887a-4ba3-88ef-2cd6ad96018b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:14.833393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317098bb-887a-4ba3-88ef-2cd6ad96018b service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:14.843050 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1307/2595] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:08:14 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:08:14.892306 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f084ef8-4e3d-4c00-a4cf-0c64165b89cb None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:14.893711 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5f084ef8-4e3d-4c00-a4cf-0c64165b89cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:14.905664 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f084ef8-4e3d-4c00-a4cf-0c64165b89cb None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:14.909341 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239704dc-7bef-44d4-9c45-e4eac3fa1ac5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:14.909934 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239704dc-7bef-44d4-9c45-e4eac3fa1ac5 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:14.910212 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239704dc-7bef-44d4-9c45-e4eac3fa1ac5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:14.910482 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239704dc-7bef-44d4-9c45-e4eac3fa1ac5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:14.970135 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239704dc-7bef-44d4-9c45-e4eac3fa1ac5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:14.971608 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239704dc-7bef-44d4-9c45-e4eac3fa1ac5 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:14.980902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1289/2596] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:08:14 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:08:19.499370 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317098bb-887a-4ba3-88ef-2cd6ad96018b service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:19.577330 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317098bb-887a-4ba3-88ef-2cd6ad96018b service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:19.581817 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f918e1-c541-4476-971e-30335bcaf6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:19.582436 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f918e1-c541-4476-971e-30335bcaf6bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:19.582743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f918e1-c541-4476-971e-30335bcaf6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:19.583016 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f918e1-c541-4476-971e-30335bcaf6bc admin admin] PATH_INFO: `/v3/users/010035bf48314776b8909bbd7eac8fd3` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:19.584120 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f918e1-c541-4476-971e-30335bcaf6bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=010035bf48314776b8909bbd7eac8fd3)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:19.586561 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f918e1-c541-4476-971e-30335bcaf6bc admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:19.733425 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-71f918e1-c541-4476-971e-30335bcaf6bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '010035bf48314776b8909bbd7eac8fd3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:19.751798 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-71f918e1-c541-4476-971e-30335bcaf6bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '010035bf48314776b8909bbd7eac8fd3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:19.753379 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-71f918e1-c541-4476-971e-30335bcaf6bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '010035bf48314776b8909bbd7eac8fd3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:19.754070 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-71f918e1-c541-4476-971e-30335bcaf6bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '010035bf48314776b8909bbd7eac8fd3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:19.758415 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:19.800094 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-71f918e1-c541-4476-971e-30335bcaf6bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '010035bf48314776b8909bbd7eac8fd3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:19.810548 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1308/2597] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:08:19 2022] DELETE /identity/v3/users/010035bf48314776b8909bbd7eac8fd3 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:19.812056 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1370f9f-d797-4060-a628-12e18fbc055a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:19.812514 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1370f9f-d797-4060-a628-12e18fbc055a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:19.812613 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1370f9f-d797-4060-a628-12e18fbc055a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:19.832866 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d1370f9f-d797-4060-a628-12e18fbc055a None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:19.834071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d1370f9f-d797-4060-a628-12e18fbc055a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:19.915506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1290/2598] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:08:19.923644 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f918e1-c541-4476-971e-30335bcaf6bc admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:19.925156 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-71f918e1-c541-4476-971e-30335bcaf6bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:19.986316 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f918e1-c541-4476-971e-30335bcaf6bc admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:19.989912 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6e655-34fc-4544-aff2-6c647c581214 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:19.990500 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6e655-34fc-4544-aff2-6c647c581214 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:19.990810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6e655-34fc-4544-aff2-6c647c581214 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:19.991077 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6e655-34fc-4544-aff2-6c647c581214 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:20.001897 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6e655-34fc-4544-aff2-6c647c581214 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:20.003377 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6e655-34fc-4544-aff2-6c647c581214 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:20.013537 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1309/2599] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:08:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:08:20.199904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1370f9f-d797-4060-a628-12e18fbc055a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:20.201752 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d1370f9f-d797-4060-a628-12e18fbc055a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:20.217264 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1370f9f-d797-4060-a628-12e18fbc055a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:20.223709 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57a635-eaa2-457b-8a20-0c66b9e90139 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:20.224070 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57a635-eaa2-457b-8a20-0c66b9e90139 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:20.224354 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57a635-eaa2-457b-8a20-0c66b9e90139 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:20.224626 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57a635-eaa2-457b-8a20-0c66b9e90139 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:20.226540 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7d57a635-eaa2-457b-8a20-0c66b9e90139 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:20.248395 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57a635-eaa2-457b-8a20-0c66b9e90139 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:20.249873 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57a635-eaa2-457b-8a20-0c66b9e90139 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:20.259309 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1291/2600] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:08:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:08:20.590247 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6e655-34fc-4544-aff2-6c647c581214 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:20.592016 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-abc6e655-34fc-4544-aff2-6c647c581214 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:20.611827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6e655-34fc-4544-aff2-6c647c581214 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:20.619486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd38bc-9826-4028-b957-5c2aa326117c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:20.620448 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd38bc-9826-4028-b957-5c2aa326117c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:20.620727 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd38bc-9826-4028-b957-5c2aa326117c admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:20.621012 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd38bc-9826-4028-b957-5c2aa326117c admin admin] PATH_INFO: `/v3/projects/09c24bf2f86b44728f6e5fdc959665ab` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:20.622313 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd38bc-9826-4028-b957-5c2aa326117c admin admin] RBAC: Authorizing `identity:delete_project(project_id=09c24bf2f86b44728f6e5fdc959665ab)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:20.624405 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd38bc-9826-4028-b957-5c2aa326117c admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:20.723928 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-74bd38bc-9826-4028-b957-5c2aa326117c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09c24bf2f86b44728f6e5fdc959665ab', 'request_id': 'req-74bd38bc-9826-4028-b957-5c2aa326117c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:20.804843 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-74bd38bc-9826-4028-b957-5c2aa326117c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09c24bf2f86b44728f6e5fdc959665ab', 'request_id': 'req-74bd38bc-9826-4028-b957-5c2aa326117c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:20.810939 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-74bd38bc-9826-4028-b957-5c2aa326117c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09c24bf2f86b44728f6e5fdc959665ab', 'request_id': 'req-74bd38bc-9826-4028-b957-5c2aa326117c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:20.822773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-74bd38bc-9826-4028-b957-5c2aa326117c admin admin] The token cache is being invalidate because project 09c24bf2f86b44728f6e5fdc959665ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:08:20.823057 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-74bd38bc-9826-4028-b957-5c2aa326117c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:20.823729 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-74bd38bc-9826-4028-b957-5c2aa326117c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:20.825202 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1310/2601] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:08:20 2022] DELETE /identity/v3/projects/09c24bf2f86b44728f6e5fdc959665ab => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:20.830221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57a635-eaa2-457b-8a20-0c66b9e90139 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:20.844361 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57a635-eaa2-457b-8a20-0c66b9e90139 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:20.883594 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:20.906770 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91724166-2225-4788-a2a4-0536661d21e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:20.907392 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91724166-2225-4788-a2a4-0536661d21e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:20.907667 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91724166-2225-4788-a2a4-0536661d21e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:20.907936 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91724166-2225-4788-a2a4-0536661d21e5 admin admin] PATH_INFO: `/v3/users/92e7bb3389c14a159c5450e3d769b123` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:20.909069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91724166-2225-4788-a2a4-0536661d21e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92e7bb3389c14a159c5450e3d769b123)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:20.911059 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91724166-2225-4788-a2a4-0536661d21e5 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:21.090429 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-91724166-2225-4788-a2a4-0536661d21e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92e7bb3389c14a159c5450e3d769b123'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:21.110182 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-91724166-2225-4788-a2a4-0536661d21e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92e7bb3389c14a159c5450e3d769b123'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:21.123689 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-91724166-2225-4788-a2a4-0536661d21e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92e7bb3389c14a159c5450e3d769b123'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:21.232680 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-91724166-2225-4788-a2a4-0536661d21e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92e7bb3389c14a159c5450e3d769b123'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:21.233281 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-91724166-2225-4788-a2a4-0536661d21e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92e7bb3389c14a159c5450e3d769b123'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:21.239523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1292/2602] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:08:20 2022] DELETE /identity/v3/users/92e7bb3389c14a159c5450e3d769b123 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:21.617426 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd38bc-9826-4028-b957-5c2aa326117c admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:21.619715 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-74bd38bc-9826-4028-b957-5c2aa326117c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:21.634230 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd38bc-9826-4028-b957-5c2aa326117c admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:21.695493 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e54425e-2b3e-4be6-99f3-75dbfd15c0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:21.696125 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54425e-2b3e-4be6-99f3-75dbfd15c0e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:21.696399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54425e-2b3e-4be6-99f3-75dbfd15c0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:21.696715 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54425e-2b3e-4be6-99f3-75dbfd15c0e7 admin admin] PATH_INFO: `/v3/projects/810da9939ba8433a8e426db6d7883de7` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:21.697820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e54425e-2b3e-4be6-99f3-75dbfd15c0e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=810da9939ba8433a8e426db6d7883de7)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:21.699803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e54425e-2b3e-4be6-99f3-75dbfd15c0e7 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:21.724707 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:21.799389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8e54425e-2b3e-4be6-99f3-75dbfd15c0e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '810da9939ba8433a8e426db6d7883de7', 'request_id': 'req-8e54425e-2b3e-4be6-99f3-75dbfd15c0e7'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:21.856299 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8e54425e-2b3e-4be6-99f3-75dbfd15c0e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '810da9939ba8433a8e426db6d7883de7', 'request_id': 'req-8e54425e-2b3e-4be6-99f3-75dbfd15c0e7'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:21.862120 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8e54425e-2b3e-4be6-99f3-75dbfd15c0e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '810da9939ba8433a8e426db6d7883de7', 'request_id': 'req-8e54425e-2b3e-4be6-99f3-75dbfd15c0e7'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:21.873466 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8e54425e-2b3e-4be6-99f3-75dbfd15c0e7 admin admin] The token cache is being invalidate because project 810da9939ba8433a8e426db6d7883de7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:08:21.873750 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8e54425e-2b3e-4be6-99f3-75dbfd15c0e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:21.874399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-8e54425e-2b3e-4be6-99f3-75dbfd15c0e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:21.876002 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1311/2603] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:08:21 2022] DELETE /identity/v3/projects/810da9939ba8433a8e426db6d7883de7 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:21.881570 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689cc6e0-c32a-4e9b-a605-142229867505 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:21.882183 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689cc6e0-c32a-4e9b-a605-142229867505 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:21.882183 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689cc6e0-c32a-4e9b-a605-142229867505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:21.903524 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-689cc6e0-c32a-4e9b-a605-142229867505 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:21.904737 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-689cc6e0-c32a-4e9b-a605-142229867505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:21.964059 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1293/2604] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:08:21.973124 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a8348-1c6e-4b74-91b4-f712d39dfdff None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:21.973407 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a8348-1c6e-4b74-91b4-f712d39dfdff None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:21.973680 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a8348-1c6e-4b74-91b4-f712d39dfdff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:21.996629 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a04a8348-1c6e-4b74-91b4-f712d39dfdff None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:21.998029 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a04a8348-1c6e-4b74-91b4-f712d39dfdff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:22.006064 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1312/2605] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:08:22.012268 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689cc6e0-c32a-4e9b-a605-142229867505 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:22.022345 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689cc6e0-c32a-4e9b-a605-142229867505 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:22.027872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e079e-3661-4d86-9abd-e52b57ca08c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:22.028471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e079e-3661-4d86-9abd-e52b57ca08c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:22.028808 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e079e-3661-4d86-9abd-e52b57ca08c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:22.029110 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e079e-3661-4d86-9abd-e52b57ca08c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:22.029759 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5e079e-3661-4d86-9abd-e52b57ca08c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:08:22.030065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5e079e-3661-4d86-9abd-e52b57ca08c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:22.031926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5e079e-3661-4d86-9abd-e52b57ca08c6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:22.044192 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1294/2606] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:08:22.048531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a8348-1c6e-4b74-91b4-f712d39dfdff None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:22.057887 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a8348-1c6e-4b74-91b4-f712d39dfdff None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:22.062584 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128774e-f367-4761-8e45-b07240faa35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:22.063190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128774e-f367-4761-8e45-b07240faa35d admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:22.063461 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128774e-f367-4761-8e45-b07240faa35d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:22.063723 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128774e-f367-4761-8e45-b07240faa35d admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:22.064355 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128774e-f367-4761-8e45-b07240faa35d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:22.066236 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128774e-f367-4761-8e45-b07240faa35d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:22.159830 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1313/2607] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:08:22 2022] POST /identity/v3/projects => generated 399 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:08:22.163946 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e079e-3661-4d86-9abd-e52b57ca08c6 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:22.179553 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e079e-3661-4d86-9abd-e52b57ca08c6 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:22.227919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:22.243013 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4db60f-a5d5-44d0-b60a-32fb023fd8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:22.243617 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4db60f-a5d5-44d0-b60a-32fb023fd8d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:22.243889 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4db60f-a5d5-44d0-b60a-32fb023fd8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:22.244153 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4db60f-a5d5-44d0-b60a-32fb023fd8d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:22.244791 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4db60f-a5d5-44d0-b60a-32fb023fd8d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:22.246719 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4db60f-a5d5-44d0-b60a-32fb023fd8d8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:22.347491 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1295/2608] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:08:22 2022] POST /identity/v3/users => generated 300 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:08:22.351759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128774e-f367-4761-8e45-b07240faa35d admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:22.362398 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128774e-f367-4761-8e45-b07240faa35d admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:22.367615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28214b3-8a34-4d39-b626-4fa831dc7341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:22.368157 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28214b3-8a34-4d39-b626-4fa831dc7341 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:22.368584 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28214b3-8a34-4d39-b626-4fa831dc7341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:22.368858 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28214b3-8a34-4d39-b626-4fa831dc7341 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:22.369495 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28214b3-8a34-4d39-b626-4fa831dc7341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:22.371222 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28214b3-8a34-4d39-b626-4fa831dc7341 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:22.381842 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1314/2609] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:08:22.386185 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4db60f-a5d5-44d0-b60a-32fb023fd8d8 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:22.396184 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4db60f-a5d5-44d0-b60a-32fb023fd8d8 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:22.401832 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873209e7-5f34-4759-b214-0c8cbeb3e999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:22.403546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873209e7-5f34-4759-b214-0c8cbeb3e999 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:22.403546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873209e7-5f34-4759-b214-0c8cbeb3e999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:22.403546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873209e7-5f34-4759-b214-0c8cbeb3e999 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:22.403852 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873209e7-5f34-4759-b214-0c8cbeb3e999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:22.405361 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873209e7-5f34-4759-b214-0c8cbeb3e999 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:22.417282 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1296/2610] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:08:22.421406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28214b3-8a34-4d39-b626-4fa831dc7341 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:22.432120 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28214b3-8a34-4d39-b626-4fa831dc7341 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:22.436833 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef3407-6c78-420f-b159-cf070907f554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:22.437491 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef3407-6c78-420f-b159-cf070907f554 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:22.437766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef3407-6c78-420f-b159-cf070907f554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:22.438097 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef3407-6c78-420f-b159-cf070907f554 admin admin] PATH_INFO: `/v3/projects/abca81513a5b43198f3d88d7dcb009aa/users/eaf6812a18da4cebad4fcfdb7cabd1e7/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:22.454297 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ef3407-6c78-420f-b159-cf070907f554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abca81513a5b43198f3d88d7dcb009aa, user_id=eaf6812a18da4cebad4fcfdb7cabd1e7, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:22.465130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ef3407-6c78-420f-b159-cf070907f554 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:22.582374 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1315/2611] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:08:22 2022] PUT /identity/v3/projects/abca81513a5b43198f3d88d7dcb009aa/users/eaf6812a18da4cebad4fcfdb7cabd1e7/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:22.593132 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620cb3e-5149-4ebc-8133-073269d2da5b None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:22.593419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620cb3e-5149-4ebc-8133-073269d2da5b None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:22.593696 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620cb3e-5149-4ebc-8133-073269d2da5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:22.621243 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f620cb3e-5149-4ebc-8133-073269d2da5b None None] MFA Rules not processed for user `eaf6812a18da4cebad4fcfdb7cabd1e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:22.622379 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f620cb3e-5149-4ebc-8133-073269d2da5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:22.686611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1297/2612] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:22 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:08:22.691317 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef3407-6c78-420f-b159-cf070907f554 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:22.755003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef3407-6c78-420f-b159-cf070907f554 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:22.760638 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f90dd-e427-4ea7-ab06-95d392f131f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:22.761525 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f90dd-e427-4ea7-ab06-95d392f131f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:22.761525 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f90dd-e427-4ea7-ab06-95d392f131f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:22.761792 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f90dd-e427-4ea7-ab06-95d392f131f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:22.762424 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9f90dd-e427-4ea7-ab06-95d392f131f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:22.764415 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9f90dd-e427-4ea7-ab06-95d392f131f6 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:22.839662 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1316/2613] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:08:22 2022] POST /identity/v3/projects => generated 401 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:08:22.843674 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620cb3e-5149-4ebc-8133-073269d2da5b None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:22.855100 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620cb3e-5149-4ebc-8133-073269d2da5b None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:22.909641 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112d8bc-c93d-4925-8696-65811afdbd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:22.910255 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112d8bc-c93d-4925-8696-65811afdbd2c admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:22.910421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112d8bc-c93d-4925-8696-65811afdbd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:22.910717 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112d8bc-c93d-4925-8696-65811afdbd2c admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:22.911347 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d112d8bc-c93d-4925-8696-65811afdbd2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:22.913787 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d112d8bc-c93d-4925-8696-65811afdbd2c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:22.984828 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1298/2614] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:08:22 2022] POST /identity/v3/users => generated 307 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:08:22.986306 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f90dd-e427-4ea7-ab06-95d392f131f6 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:22.997353 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f90dd-e427-4ea7-ab06-95d392f131f6 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:23.001955 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be0815-28a4-4c72-866f-2955659fa49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:23.002541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be0815-28a4-4c72-866f-2955659fa49e admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:23.002867 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be0815-28a4-4c72-866f-2955659fa49e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:23.003128 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be0815-28a4-4c72-866f-2955659fa49e admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:23.003786 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be0815-28a4-4c72-866f-2955659fa49e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:23.005393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be0815-28a4-4c72-866f-2955659fa49e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:23.016114 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1317/2615] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:08:23.020270 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112d8bc-c93d-4925-8696-65811afdbd2c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:23.030745 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112d8bc-c93d-4925-8696-65811afdbd2c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:23.035451 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf6794-4a03-4335-8ab0-2358eb3a784e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:23.036097 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf6794-4a03-4335-8ab0-2358eb3a784e admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:23.036375 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf6794-4a03-4335-8ab0-2358eb3a784e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:23.037463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf6794-4a03-4335-8ab0-2358eb3a784e admin admin] PATH_INFO: `/v3/projects/73a5184ff8404886858c4c730d0d6398/users/8d869c16e7e0415e9bc2445236e4eba4/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:23.053796 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bf6794-4a03-4335-8ab0-2358eb3a784e admin admin] RBAC: Authorizing `identity:create_grant(project_id=73a5184ff8404886858c4c730d0d6398, user_id=8d869c16e7e0415e9bc2445236e4eba4, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:23.057284 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bf6794-4a03-4335-8ab0-2358eb3a784e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:23.147836 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1299/2616] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:08:23 2022] PUT /identity/v3/projects/73a5184ff8404886858c4c730d0d6398/users/8d869c16e7e0415e9bc2445236e4eba4/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:08:23.152015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be0815-28a4-4c72-866f-2955659fa49e admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:23.162066 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be0815-28a4-4c72-866f-2955659fa49e admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:23.181601 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:23.219571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6213db-7deb-438f-8c3d-fb13562d7292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:23.220158 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6213db-7deb-438f-8c3d-fb13562d7292 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:23.220437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6213db-7deb-438f-8c3d-fb13562d7292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:23.220705 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6213db-7deb-438f-8c3d-fb13562d7292 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:23.221336 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6213db-7deb-438f-8c3d-fb13562d7292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:23.223247 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6213db-7deb-438f-8c3d-fb13562d7292 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:23.237576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1318/2617] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:08:23.240067 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf6794-4a03-4335-8ab0-2358eb3a784e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:23.250824 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf6794-4a03-4335-8ab0-2358eb3a784e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:23.255701 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ed346-7b9d-4d98-a14c-ae8e7e4d32e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:23.256436 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ed346-7b9d-4d98-a14c-ae8e7e4d32e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:23.256796 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ed346-7b9d-4d98-a14c-ae8e7e4d32e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:23.257203 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ed346-7b9d-4d98-a14c-ae8e7e4d32e8 admin admin] PATH_INFO: `/v3/projects/73a5184ff8404886858c4c730d0d6398/users/8d869c16e7e0415e9bc2445236e4eba4/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:23.259671 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ed346-7b9d-4d98-a14c-ae8e7e4d32e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73a5184ff8404886858c4c730d0d6398, user_id=8d869c16e7e0415e9bc2445236e4eba4, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:23.262017 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ed346-7b9d-4d98-a14c-ae8e7e4d32e8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:23.273205 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1300/2618] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:08:23 2022] PUT /identity/v3/projects/73a5184ff8404886858c4c730d0d6398/users/8d869c16e7e0415e9bc2445236e4eba4/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:08:23.284817 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18811dcb-d327-420a-a0e1-96d0aaa4e662 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:23.285099 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18811dcb-d327-420a-a0e1-96d0aaa4e662 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:23.285373 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18811dcb-d327-420a-a0e1-96d0aaa4e662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:23.315677 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-18811dcb-d327-420a-a0e1-96d0aaa4e662 None None] MFA Rules not processed for user `8d869c16e7e0415e9bc2445236e4eba4`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:23.316924 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-18811dcb-d327-420a-a0e1-96d0aaa4e662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:23.393877 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1319/2619] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:23 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:08:23.405702 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ed346-7b9d-4d98-a14c-ae8e7e4d32e8 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:23.407517 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c53ed346-7b9d-4d98-a14c-ae8e7e4d32e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:23.415649 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:23.468158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ed346-7b9d-4d98-a14c-ae8e7e4d32e8 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:23.472744 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44e98d-a0b4-4022-871c-5206343e9999 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:23.473600 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44e98d-a0b4-4022-871c-5206343e9999 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:23.473903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44e98d-a0b4-4022-871c-5206343e9999 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:23.474310 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44e98d-a0b4-4022-871c-5206343e9999 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:23.484481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e44e98d-a0b4-4022-871c-5206343e9999 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:23.486073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e44e98d-a0b4-4022-871c-5206343e9999 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:23.497835 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1301/2620] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:08:23 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:08:23.687710 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18811dcb-d327-420a-a0e1-96d0aaa4e662 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:23.698055 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18811dcb-d327-420a-a0e1-96d0aaa4e662 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:23.702226 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbccdfb-51e2-4c8d-ba92-e310048a1d9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:23.702936 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbccdfb-51e2-4c8d-ba92-e310048a1d9a service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:23.703212 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbccdfb-51e2-4c8d-ba92-e310048a1d9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:23.703480 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbccdfb-51e2-4c8d-ba92-e310048a1d9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:23.764363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbccdfb-51e2-4c8d-ba92-e310048a1d9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:23.765932 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbccdfb-51e2-4c8d-ba92-e310048a1d9a service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:23.775180 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1320/2621] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:08:23 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:08:28.809247 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44e98d-a0b4-4022-871c-5206343e9999 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:28.873679 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44e98d-a0b4-4022-871c-5206343e9999 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:28.878593 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba4829-5ee5-4c06-8f7a-61634088b07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:28.879245 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba4829-5ee5-4c06-8f7a-61634088b07d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:28.879644 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba4829-5ee5-4c06-8f7a-61634088b07d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:28.879838 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba4829-5ee5-4c06-8f7a-61634088b07d admin admin] PATH_INFO: `/v3/users/eaf6812a18da4cebad4fcfdb7cabd1e7` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:28.880870 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba4829-5ee5-4c06-8f7a-61634088b07d admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaf6812a18da4cebad4fcfdb7cabd1e7)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:28.883679 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba4829-5ee5-4c06-8f7a-61634088b07d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:29.061248 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-68ba4829-5ee5-4c06-8f7a-61634088b07d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaf6812a18da4cebad4fcfdb7cabd1e7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:29.069534 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-68ba4829-5ee5-4c06-8f7a-61634088b07d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaf6812a18da4cebad4fcfdb7cabd1e7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:29.082646 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-68ba4829-5ee5-4c06-8f7a-61634088b07d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaf6812a18da4cebad4fcfdb7cabd1e7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:29.158450 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-68ba4829-5ee5-4c06-8f7a-61634088b07d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaf6812a18da4cebad4fcfdb7cabd1e7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:29.159117 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-68ba4829-5ee5-4c06-8f7a-61634088b07d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaf6812a18da4cebad4fcfdb7cabd1e7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:29.169448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1302/2622] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:08:28 2022] DELETE /identity/v3/users/eaf6812a18da4cebad4fcfdb7cabd1e7 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:29.172322 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbefb2-6222-4124-a4d4-756c90fe8eb8 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:29.172322 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbefb2-6222-4124-a4d4-756c90fe8eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:29.172876 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbefb2-6222-4124-a4d4-756c90fe8eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:29.197916 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-8abbefb2-6222-4124-a4d4-756c90fe8eb8 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:29.199543 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8abbefb2-6222-4124-a4d4-756c90fe8eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:29.266584 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=100 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:29.298475 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1321/2623] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:08:29.323830 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba4829-5ee5-4c06-8f7a-61634088b07d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:29.325429 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-68ba4829-5ee5-4c06-8f7a-61634088b07d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:29.367817 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:29.395705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba4829-5ee5-4c06-8f7a-61634088b07d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:29.399878 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f46166-d53a-4392-91c8-bd65d342d543 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:29.400571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f46166-d53a-4392-91c8-bd65d342d543 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:29.400937 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f46166-d53a-4392-91c8-bd65d342d543 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:29.401277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f46166-d53a-4392-91c8-bd65d342d543 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:29.412847 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f46166-d53a-4392-91c8-bd65d342d543 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:29.414930 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f46166-d53a-4392-91c8-bd65d342d543 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:29.430770 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1303/2624] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:08:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:08:29.863179 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abbefb2-6222-4124-a4d4-756c90fe8eb8 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:29.866372 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-8abbefb2-6222-4124-a4d4-756c90fe8eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:29.877440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abbefb2-6222-4124-a4d4-756c90fe8eb8 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:29.882806 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adee8f2-3a3a-4f05-af05-64f83bb58c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:29.883451 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adee8f2-3a3a-4f05-af05-64f83bb58c67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:29.883727 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adee8f2-3a3a-4f05-af05-64f83bb58c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:29.884004 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adee8f2-3a3a-4f05-af05-64f83bb58c67 admin admin] PATH_INFO: `/v3/projects/abca81513a5b43198f3d88d7dcb009aa` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:29.885134 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adee8f2-3a3a-4f05-af05-64f83bb58c67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abca81513a5b43198f3d88d7dcb009aa)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:29.887101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adee8f2-3a3a-4f05-af05-64f83bb58c67 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:30.028212 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-9adee8f2-3a3a-4f05-af05-64f83bb58c67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abca81513a5b43198f3d88d7dcb009aa', 'request_id': 'req-9adee8f2-3a3a-4f05-af05-64f83bb58c67'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:30.129732 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-9adee8f2-3a3a-4f05-af05-64f83bb58c67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abca81513a5b43198f3d88d7dcb009aa', 'request_id': 'req-9adee8f2-3a3a-4f05-af05-64f83bb58c67'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:30.136543 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-9adee8f2-3a3a-4f05-af05-64f83bb58c67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abca81513a5b43198f3d88d7dcb009aa', 'request_id': 'req-9adee8f2-3a3a-4f05-af05-64f83bb58c67'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:30.147878 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-9adee8f2-3a3a-4f05-af05-64f83bb58c67 admin admin] The token cache is being invalidate because project abca81513a5b43198f3d88d7dcb009aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:08:30.148256 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-9adee8f2-3a3a-4f05-af05-64f83bb58c67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:30.148899 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-9adee8f2-3a3a-4f05-af05-64f83bb58c67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:30.150385 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1322/2625] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:08:29 2022] DELETE /identity/v3/projects/abca81513a5b43198f3d88d7dcb009aa => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:30.154368 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f46166-d53a-4392-91c8-bd65d342d543 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:30.163907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f46166-d53a-4392-91c8-bd65d342d543 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:30.220149 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb91e4-7732-4f06-b6ae-86797634c085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:30.220769 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb91e4-7732-4f06-b6ae-86797634c085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:30.221047 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb91e4-7732-4f06-b6ae-86797634c085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:30.221320 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb91e4-7732-4f06-b6ae-86797634c085 admin admin] PATH_INFO: `/v3/users/8d869c16e7e0415e9bc2445236e4eba4` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:30.222429 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb91e4-7732-4f06-b6ae-86797634c085 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d869c16e7e0415e9bc2445236e4eba4)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:30.224387 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb91e4-7732-4f06-b6ae-86797634c085 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:30.416361 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2dfb91e4-7732-4f06-b6ae-86797634c085 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d869c16e7e0415e9bc2445236e4eba4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:30.423767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2dfb91e4-7732-4f06-b6ae-86797634c085 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d869c16e7e0415e9bc2445236e4eba4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:30.436810 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2dfb91e4-7732-4f06-b6ae-86797634c085 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d869c16e7e0415e9bc2445236e4eba4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:30.513630 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2dfb91e4-7732-4f06-b6ae-86797634c085 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d869c16e7e0415e9bc2445236e4eba4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:30.514348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-2dfb91e4-7732-4f06-b6ae-86797634c085 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d869c16e7e0415e9bc2445236e4eba4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:30.523769 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1304/2626] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:08:30 2022] DELETE /identity/v3/users/8d869c16e7e0415e9bc2445236e4eba4 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:31.023824 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adee8f2-3a3a-4f05-af05-64f83bb58c67 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:31.025385 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-9adee8f2-3a3a-4f05-af05-64f83bb58c67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:31.039999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adee8f2-3a3a-4f05-af05-64f83bb58c67 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:31.101617 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce69dd-2c95-4e40-baf3-38cbe72e7bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:31.102177 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce69dd-2c95-4e40-baf3-38cbe72e7bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:31.102456 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce69dd-2c95-4e40-baf3-38cbe72e7bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:31.102754 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce69dd-2c95-4e40-baf3-38cbe72e7bd0 admin admin] PATH_INFO: `/v3/projects/73a5184ff8404886858c4c730d0d6398` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:31.103879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce69dd-2c95-4e40-baf3-38cbe72e7bd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73a5184ff8404886858c4c730d0d6398)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:31.105791 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce69dd-2c95-4e40-baf3-38cbe72e7bd0 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:31.219573 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-09ce69dd-2c95-4e40-baf3-38cbe72e7bd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73a5184ff8404886858c4c730d0d6398', 'request_id': 'req-09ce69dd-2c95-4e40-baf3-38cbe72e7bd0'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:31.290241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-09ce69dd-2c95-4e40-baf3-38cbe72e7bd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73a5184ff8404886858c4c730d0d6398', 'request_id': 'req-09ce69dd-2c95-4e40-baf3-38cbe72e7bd0'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:31.296767 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-09ce69dd-2c95-4e40-baf3-38cbe72e7bd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73a5184ff8404886858c4c730d0d6398', 'request_id': 'req-09ce69dd-2c95-4e40-baf3-38cbe72e7bd0'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:31.298689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:31.309452 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-09ce69dd-2c95-4e40-baf3-38cbe72e7bd0 admin admin] The token cache is being invalidate because project 73a5184ff8404886858c4c730d0d6398 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:08:31.309736 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-09ce69dd-2c95-4e40-baf3-38cbe72e7bd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:31.310380 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-09ce69dd-2c95-4e40-baf3-38cbe72e7bd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:31.311873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1323/2627] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:08:31 2022] DELETE /identity/v3/projects/73a5184ff8404886858c4c730d0d6398 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:31.316661 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352b730-14b7-489d-ad62-096d67f7934e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:31.317065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352b730-14b7-489d-ad62-096d67f7934e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:31.317281 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352b730-14b7-489d-ad62-096d67f7934e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:31.338100 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5352b730-14b7-489d-ad62-096d67f7934e None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:31.339463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5352b730-14b7-489d-ad62-096d67f7934e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:31.385343 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:31.400900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1305/2628] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:08:31.408715 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b29ae7-2574-4983-a874-e4c59dd1d9ad None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:31.408996 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b29ae7-2574-4983-a874-e4c59dd1d9ad None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:31.409264 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b29ae7-2574-4983-a874-e4c59dd1d9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:31.429934 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-98b29ae7-2574-4983-a874-e4c59dd1d9ad None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:31.431252 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-98b29ae7-2574-4983-a874-e4c59dd1d9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:31.439313 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1324/2629] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:08:31.449263 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352b730-14b7-489d-ad62-096d67f7934e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:31.454110 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352b730-14b7-489d-ad62-096d67f7934e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:31.459302 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae102545-e21e-4dbd-99fe-ba40f431a6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:31.459914 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae102545-e21e-4dbd-99fe-ba40f431a6ac admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:31.460690 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae102545-e21e-4dbd-99fe-ba40f431a6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:31.461003 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae102545-e21e-4dbd-99fe-ba40f431a6ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:31.461654 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae102545-e21e-4dbd-99fe-ba40f431a6ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:08:31.462089 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae102545-e21e-4dbd-99fe-ba40f431a6ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:31.463808 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae102545-e21e-4dbd-99fe-ba40f431a6ac admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:31.477448 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1306/2630] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:08:31.481911 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b29ae7-2574-4983-a874-e4c59dd1d9ad None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:31.491174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b29ae7-2574-4983-a874-e4c59dd1d9ad None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:31.496662 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594d454-575f-4c87-a262-c672cd1d2762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:31.499870 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594d454-575f-4c87-a262-c672cd1d2762 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:31.499870 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594d454-575f-4c87-a262-c672cd1d2762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:31.499870 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594d454-575f-4c87-a262-c672cd1d2762 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:31.499870 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b594d454-575f-4c87-a262-c672cd1d2762 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:31.500449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b594d454-575f-4c87-a262-c672cd1d2762 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:31.589576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1325/2631] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:08:31 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:08:31.594229 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae102545-e21e-4dbd-99fe-ba40f431a6ac admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:31.604930 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae102545-e21e-4dbd-99fe-ba40f431a6ac admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:31.661018 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4af20-c448-4d7d-b5ff-2a6db179c304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:31.661622 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4af20-c448-4d7d-b5ff-2a6db179c304 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:31.662333 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4af20-c448-4d7d-b5ff-2a6db179c304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:31.663164 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4af20-c448-4d7d-b5ff-2a6db179c304 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:31.663794 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4af20-c448-4d7d-b5ff-2a6db179c304 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:31.665782 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4af20-c448-4d7d-b5ff-2a6db179c304 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:31.760518 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1307/2632] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:08:31 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:08:31.765777 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594d454-575f-4c87-a262-c672cd1d2762 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:31.776804 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594d454-575f-4c87-a262-c672cd1d2762 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:31.781404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9720bc-a744-4855-b527-b8bc08ca3eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:31.782000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9720bc-a744-4855-b527-b8bc08ca3eee admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:31.782276 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9720bc-a744-4855-b527-b8bc08ca3eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:31.782547 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9720bc-a744-4855-b527-b8bc08ca3eee admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:31.783220 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9720bc-a744-4855-b527-b8bc08ca3eee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:31.784804 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9720bc-a744-4855-b527-b8bc08ca3eee admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:31.795153 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1326/2633] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:08:31.799312 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4af20-c448-4d7d-b5ff-2a6db179c304 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:31.809887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4af20-c448-4d7d-b5ff-2a6db179c304 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:31.814595 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d3fb9-b353-4e45-9385-f779013abc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:31.815209 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d3fb9-b353-4e45-9385-f779013abc08 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:31.815482 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d3fb9-b353-4e45-9385-f779013abc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:31.815749 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d3fb9-b353-4e45-9385-f779013abc08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:31.816377 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d3fb9-b353-4e45-9385-f779013abc08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:31.818012 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d3fb9-b353-4e45-9385-f779013abc08 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:31.829226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1308/2634] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:08:31.833372 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9720bc-a744-4855-b527-b8bc08ca3eee admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:31.843807 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9720bc-a744-4855-b527-b8bc08ca3eee admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:31.848501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10336321-523e-49fe-bc72-0ec5a26d6bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:31.849137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10336321-523e-49fe-bc72-0ec5a26d6bf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:31.849412 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10336321-523e-49fe-bc72-0ec5a26d6bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:31.849683 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10336321-523e-49fe-bc72-0ec5a26d6bf7 admin admin] PATH_INFO: `/v3/projects/cc99c2c20a0346998ee35ae8aadb6e77/users/20743ace20ce4954b62b6380ac09117e/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:31.864809 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10336321-523e-49fe-bc72-0ec5a26d6bf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc99c2c20a0346998ee35ae8aadb6e77, user_id=20743ace20ce4954b62b6380ac09117e, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:31.867520 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10336321-523e-49fe-bc72-0ec5a26d6bf7 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:31.953504 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1327/2635] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:08:31 2022] PUT /identity/v3/projects/cc99c2c20a0346998ee35ae8aadb6e77/users/20743ace20ce4954b62b6380ac09117e/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:31.965017 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c2810-5ed3-40c0-9c6f-cfe036cbc546 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:31.965348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c2810-5ed3-40c0-9c6f-cfe036cbc546 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:31.965625 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c2810-5ed3-40c0-9c6f-cfe036cbc546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:31.993487 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d71c2810-5ed3-40c0-9c6f-cfe036cbc546 None None] MFA Rules not processed for user `20743ace20ce4954b62b6380ac09117e`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:31.994725 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d71c2810-5ed3-40c0-9c6f-cfe036cbc546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:32.057903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1309/2636] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:08:32.062770 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10336321-523e-49fe-bc72-0ec5a26d6bf7 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:32.131180 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10336321-523e-49fe-bc72-0ec5a26d6bf7 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:32.136600 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8c323-cde4-440d-b34d-1eb3d8891e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:32.137199 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8c323-cde4-440d-b34d-1eb3d8891e24 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:32.137473 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8c323-cde4-440d-b34d-1eb3d8891e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:32.137745 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8c323-cde4-440d-b34d-1eb3d8891e24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:32.138378 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8c323-cde4-440d-b34d-1eb3d8891e24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:32.140372 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8c323-cde4-440d-b34d-1eb3d8891e24 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:32.228889 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1328/2637] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:08:32 2022] POST /identity/v3/projects => generated 377 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:08:32.233030 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c2810-5ed3-40c0-9c6f-cfe036cbc546 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:32.244303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c2810-5ed3-40c0-9c6f-cfe036cbc546 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:32.285224 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:32.299617 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db6018-cca4-4200-ac2a-18758da1fad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:32.300219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db6018-cca4-4200-ac2a-18758da1fad7 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:32.300495 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db6018-cca4-4200-ac2a-18758da1fad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:32.300765 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db6018-cca4-4200-ac2a-18758da1fad7 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:32.301405 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db6018-cca4-4200-ac2a-18758da1fad7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:32.303355 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db6018-cca4-4200-ac2a-18758da1fad7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:32.399064 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1310/2638] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:08:32 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:08:32.403080 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8c323-cde4-440d-b34d-1eb3d8891e24 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:32.413071 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8c323-cde4-440d-b34d-1eb3d8891e24 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:32.418493 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db878e-ecb6-40e5-91b6-d5620193376e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:32.419113 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db878e-ecb6-40e5-91b6-d5620193376e admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:32.419387 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db878e-ecb6-40e5-91b6-d5620193376e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:32.419670 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db878e-ecb6-40e5-91b6-d5620193376e admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:32.420297 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db878e-ecb6-40e5-91b6-d5620193376e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:32.421865 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db878e-ecb6-40e5-91b6-d5620193376e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:32.433414 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1329/2639] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:08:32.437646 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db6018-cca4-4200-ac2a-18758da1fad7 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:32.448472 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db6018-cca4-4200-ac2a-18758da1fad7 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:32.453325 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af0ac6-b3e7-48a2-8466-60eecbc7aaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:32.453913 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af0ac6-b3e7-48a2-8466-60eecbc7aaff admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:32.454193 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af0ac6-b3e7-48a2-8466-60eecbc7aaff admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:32.454467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af0ac6-b3e7-48a2-8466-60eecbc7aaff admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:32.455145 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af0ac6-b3e7-48a2-8466-60eecbc7aaff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:32.456775 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af0ac6-b3e7-48a2-8466-60eecbc7aaff admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:32.466644 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1311/2640] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:08:32.470739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db878e-ecb6-40e5-91b6-d5620193376e admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:32.480708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db878e-ecb6-40e5-91b6-d5620193376e admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:32.485342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863d145-3d8a-47cf-ae06-fab233e54f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:32.486023 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863d145-3d8a-47cf-ae06-fab233e54f4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:32.486304 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863d145-3d8a-47cf-ae06-fab233e54f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:32.486579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863d145-3d8a-47cf-ae06-fab233e54f4d admin admin] PATH_INFO: `/v3/projects/3d8078fbb994454ba5f4c344ff647859/users/5f818b3fa9864108902889acfdb36709/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:32.502090 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863d145-3d8a-47cf-ae06-fab233e54f4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d8078fbb994454ba5f4c344ff647859, user_id=5f818b3fa9864108902889acfdb36709, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:32.504759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863d145-3d8a-47cf-ae06-fab233e54f4d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:32.582617 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1330/2641] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:08:32 2022] PUT /identity/v3/projects/3d8078fbb994454ba5f4c344ff647859/users/5f818b3fa9864108902889acfdb36709/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:32.593890 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b782d1-d801-47f9-8d6c-327c9ab5b573 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:32.594173 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b782d1-d801-47f9-8d6c-327c9ab5b573 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:32.594444 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b782d1-d801-47f9-8d6c-327c9ab5b573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:32.630942 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-24b782d1-d801-47f9-8d6c-327c9ab5b573 None None] MFA Rules not processed for user `5f818b3fa9864108902889acfdb36709`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:32.632300 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-24b782d1-d801-47f9-8d6c-327c9ab5b573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:32.695323 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1312/2642] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:08:32.699735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863d145-3d8a-47cf-ae06-fab233e54f4d admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:32.746253 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:32.763863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863d145-3d8a-47cf-ae06-fab233e54f4d admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:32.768618 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84b3f4-678e-4b51-b077-d406583072af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:32.769220 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84b3f4-678e-4b51-b077-d406583072af admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:32.769493 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84b3f4-678e-4b51-b077-d406583072af admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:32.771292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84b3f4-678e-4b51-b077-d406583072af admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:32.771292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84b3f4-678e-4b51-b077-d406583072af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:32.772279 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84b3f4-678e-4b51-b077-d406583072af admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:32.862996 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1331/2643] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:08:32 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:08:32.867661 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b782d1-d801-47f9-8d6c-327c9ab5b573 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:32.878843 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b782d1-d801-47f9-8d6c-327c9ab5b573 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:32.938976 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6635fe73-f58b-436a-b379-2a05451e43ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:32.939612 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635fe73-f58b-436a-b379-2a05451e43ef admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:32.939887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635fe73-f58b-436a-b379-2a05451e43ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:32.940160 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635fe73-f58b-436a-b379-2a05451e43ef admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:32.940789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6635fe73-f58b-436a-b379-2a05451e43ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:32.942846 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6635fe73-f58b-436a-b379-2a05451e43ef admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:33.032825 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1313/2644] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:08:32 2022] POST /identity/v3/users => generated 295 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:08:33.036928 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84b3f4-678e-4b51-b077-d406583072af admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:33.047256 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84b3f4-678e-4b51-b077-d406583072af admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:33.052132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f50da-9c73-4df0-a600-7ad684cdafa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:33.052784 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f50da-9c73-4df0-a600-7ad684cdafa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:33.053061 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f50da-9c73-4df0-a600-7ad684cdafa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:33.053328 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f50da-9c73-4df0-a600-7ad684cdafa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:33.055119 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f50da-9c73-4df0-a600-7ad684cdafa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:33.056713 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f50da-9c73-4df0-a600-7ad684cdafa7 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:33.066870 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1332/2645] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:08:33.071684 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6635fe73-f58b-436a-b379-2a05451e43ef admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:33.082177 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6635fe73-f58b-436a-b379-2a05451e43ef admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:33.087172 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a0537-4555-486f-ab11-64b2a138c99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:33.087818 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a0537-4555-486f-ab11-64b2a138c99c admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:33.088097 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a0537-4555-486f-ab11-64b2a138c99c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:33.088368 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a0537-4555-486f-ab11-64b2a138c99c admin admin] PATH_INFO: `/v3/projects/f1080d461e1f4ba48df112a1e3b1bcae/users/1a0816887b69416389458fb6d0309213/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:33.103269 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369a0537-4555-486f-ab11-64b2a138c99c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1080d461e1f4ba48df112a1e3b1bcae, user_id=1a0816887b69416389458fb6d0309213, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:33.105951 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369a0537-4555-486f-ab11-64b2a138c99c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:33.193075 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1314/2646] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:08:33 2022] PUT /identity/v3/projects/f1080d461e1f4ba48df112a1e3b1bcae/users/1a0816887b69416389458fb6d0309213/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:33.197856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f50da-9c73-4df0-a600-7ad684cdafa7 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:33.208026 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f50da-9c73-4df0-a600-7ad684cdafa7 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:33.264212 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e0fe3-8ffe-403b-92a3-deeb6f9e6068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:33.264803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e0fe3-8ffe-403b-92a3-deeb6f9e6068 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:33.265074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e0fe3-8ffe-403b-92a3-deeb6f9e6068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:33.265343 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e0fe3-8ffe-403b-92a3-deeb6f9e6068 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:33.265969 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415e0fe3-8ffe-403b-92a3-deeb6f9e6068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:33.267571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415e0fe3-8ffe-403b-92a3-deeb6f9e6068 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:33.277834 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1333/2647] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:08:33.281871 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a0537-4555-486f-ab11-64b2a138c99c admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:33.292978 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a0537-4555-486f-ab11-64b2a138c99c admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:33.297477 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35469c-4f52-4a5b-83a9-7a78470e7ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:33.298197 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35469c-4f52-4a5b-83a9-7a78470e7ae0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:33.298472 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35469c-4f52-4a5b-83a9-7a78470e7ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:33.298779 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35469c-4f52-4a5b-83a9-7a78470e7ae0 admin admin] PATH_INFO: `/v3/projects/f1080d461e1f4ba48df112a1e3b1bcae/users/1a0816887b69416389458fb6d0309213/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:33.300711 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35469c-4f52-4a5b-83a9-7a78470e7ae0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1080d461e1f4ba48df112a1e3b1bcae, user_id=1a0816887b69416389458fb6d0309213, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:33.303431 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35469c-4f52-4a5b-83a9-7a78470e7ae0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:33.315583 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1315/2648] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:08:33 2022] PUT /identity/v3/projects/f1080d461e1f4ba48df112a1e3b1bcae/users/1a0816887b69416389458fb6d0309213/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:08:33.326910 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d4d26-a509-45c4-8d0d-e8edeb6f3612 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:33.327196 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d4d26-a509-45c4-8d0d-e8edeb6f3612 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:33.327469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d4d26-a509-45c4-8d0d-e8edeb6f3612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:33.355407 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a93d4d26-a509-45c4-8d0d-e8edeb6f3612 None None] MFA Rules not processed for user `1a0816887b69416389458fb6d0309213`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:33.356623 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a93d4d26-a509-45c4-8d0d-e8edeb6f3612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:33.424511 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1334/2649] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:08:33.436789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35469c-4f52-4a5b-83a9-7a78470e7ae0 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:33.438960 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ce35469c-4f52-4a5b-83a9-7a78470e7ae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:33.445777 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:33.499875 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35469c-4f52-4a5b-83a9-7a78470e7ae0 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:33.503692 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f78cf6-56b6-45f6-ac33-d7583cea239b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:33.504319 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f78cf6-56b6-45f6-ac33-d7583cea239b service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:33.504599 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f78cf6-56b6-45f6-ac33-d7583cea239b service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:33.504898 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f78cf6-56b6-45f6-ac33-d7583cea239b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:33.513901 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f78cf6-56b6-45f6-ac33-d7583cea239b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:33.515379 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f78cf6-56b6-45f6-ac33-d7583cea239b service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:33.524711 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1316/2650] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:08:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:08:33.636931 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d4d26-a509-45c4-8d0d-e8edeb6f3612 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:33.638348 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a93d4d26-a509-45c4-8d0d-e8edeb6f3612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:33.650169 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d4d26-a509-45c4-8d0d-e8edeb6f3612 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:33.653882 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d796f27-8175-475a-bc15-4c8ba57fcd0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:33.654559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d796f27-8175-475a-bc15-4c8ba57fcd0b service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:33.654859 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d796f27-8175-475a-bc15-4c8ba57fcd0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:33.655143 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d796f27-8175-475a-bc15-4c8ba57fcd0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:33.689061 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:33.712514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d796f27-8175-475a-bc15-4c8ba57fcd0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:33.713975 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d796f27-8175-475a-bc15-4c8ba57fcd0b service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:33.722997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1335/2651] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:08:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:08:33.950527 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f78cf6-56b6-45f6-ac33-d7583cea239b service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:33.952425 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c4f78cf6-56b6-45f6-ac33-d7583cea239b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:34.011984 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f78cf6-56b6-45f6-ac33-d7583cea239b service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:34.015602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8265b9-cb98-4b4f-b271-1c355c82df3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:34.016234 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8265b9-cb98-4b4f-b271-1c355c82df3a service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:34.016505 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8265b9-cb98-4b4f-b271-1c355c82df3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:34.016772 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8265b9-cb98-4b4f-b271-1c355c82df3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:34.018476 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3e8265b9-cb98-4b4f-b271-1c355c82df3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:34.119075 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8265b9-cb98-4b4f-b271-1c355c82df3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:34.120595 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8265b9-cb98-4b4f-b271-1c355c82df3a service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:34.132934 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1317/2652] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:08:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:08:37.749599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d796f27-8175-475a-bc15-4c8ba57fcd0b service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:37.764775 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d796f27-8175-475a-bc15-4c8ba57fcd0b service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:37.780174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821afe7-ca89-4afe-a54d-25fb2b8b2026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:37.780828 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821afe7-ca89-4afe-a54d-25fb2b8b2026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:37.781103 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821afe7-ca89-4afe-a54d-25fb2b8b2026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:37.781371 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821afe7-ca89-4afe-a54d-25fb2b8b2026 admin admin] PATH_INFO: `/v3/users/20743ace20ce4954b62b6380ac09117e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:37.782529 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f821afe7-ca89-4afe-a54d-25fb2b8b2026 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20743ace20ce4954b62b6380ac09117e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:37.784469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f821afe7-ca89-4afe-a54d-25fb2b8b2026 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:37.958502 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f821afe7-ca89-4afe-a54d-25fb2b8b2026 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20743ace20ce4954b62b6380ac09117e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:37.987252 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f821afe7-ca89-4afe-a54d-25fb2b8b2026 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20743ace20ce4954b62b6380ac09117e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:37.994864 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f821afe7-ca89-4afe-a54d-25fb2b8b2026 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20743ace20ce4954b62b6380ac09117e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:37.995663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f821afe7-ca89-4afe-a54d-25fb2b8b2026 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20743ace20ce4954b62b6380ac09117e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:38.064714 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8265b9-cb98-4b4f-b271-1c355c82df3a service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:38.066215 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-3e8265b9-cb98-4b4f-b271-1c355c82df3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:38.072938 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f821afe7-ca89-4afe-a54d-25fb2b8b2026 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20743ace20ce4954b62b6380ac09117e'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:38.080187 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8265b9-cb98-4b4f-b271-1c355c82df3a service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:38.081292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1336/2653] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:08:37 2022] DELETE /identity/v3/users/20743ace20ce4954b62b6380ac09117e => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:38.086072 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe6e1a-27fe-43f1-90ab-9e6e2193ab4c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:38.086284 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe6e1a-27fe-43f1-90ab-9e6e2193ab4c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:38.086553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe6e1a-27fe-43f1-90ab-9e6e2193ab4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:38.106243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-14fe6e1a-27fe-43f1-90ab-9e6e2193ab4c None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:38.107493 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-14fe6e1a-27fe-43f1-90ab-9e6e2193ab4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:38.135564 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:38.141449 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31a9d4-54e7-44e4-b6fa-9d60c13e0a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:38.142071 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31a9d4-54e7-44e4-b6fa-9d60c13e0a6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:38.142346 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31a9d4-54e7-44e4-b6fa-9d60c13e0a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:38.142615 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31a9d4-54e7-44e4-b6fa-9d60c13e0a6f admin admin] PATH_INFO: `/v3/users/a66188ce11bc403d9c657efd78f3019a` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:38.143744 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31a9d4-54e7-44e4-b6fa-9d60c13e0a6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a66188ce11bc403d9c657efd78f3019a)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:38.145696 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31a9d4-54e7-44e4-b6fa-9d60c13e0a6f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:38.167678 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1337/2654] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:08:38.178803 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe6e1a-27fe-43f1-90ab-9e6e2193ab4c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:38.180703 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-14fe6e1a-27fe-43f1-90ab-9e6e2193ab4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:38.234116 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:38.237898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe6e1a-27fe-43f1-90ab-9e6e2193ab4c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:38.242095 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ca292-5764-4ed4-906b-96214963e117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:38.242886 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ca292-5764-4ed4-906b-96214963e117 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:38.243176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ca292-5764-4ed4-906b-96214963e117 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:38.243438 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ca292-5764-4ed4-906b-96214963e117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:38.252481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ca292-5764-4ed4-906b-96214963e117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:38.253906 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ca292-5764-4ed4-906b-96214963e117 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:38.264102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1338/2655] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:08:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:08:38.299901 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dc31a9d4-54e7-44e4-b6fa-9d60c13e0a6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a66188ce11bc403d9c657efd78f3019a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:38.313433 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dc31a9d4-54e7-44e4-b6fa-9d60c13e0a6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a66188ce11bc403d9c657efd78f3019a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:38.326981 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dc31a9d4-54e7-44e4-b6fa-9d60c13e0a6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a66188ce11bc403d9c657efd78f3019a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:38.409968 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dc31a9d4-54e7-44e4-b6fa-9d60c13e0a6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a66188ce11bc403d9c657efd78f3019a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:38.410600 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-dc31a9d4-54e7-44e4-b6fa-9d60c13e0a6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a66188ce11bc403d9c657efd78f3019a'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:38.417929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1318/2656] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:08:38 2022] DELETE /identity/v3/users/a66188ce11bc403d9c657efd78f3019a => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:08:38.719774 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31a9d4-54e7-44e4-b6fa-9d60c13e0a6f admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:38.719994 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ca292-5764-4ed4-906b-96214963e117 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:38.720124 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-dc31a9d4-54e7-44e4-b6fa-9d60c13e0a6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:38.720559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-592ca292-5764-4ed4-906b-96214963e117 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:38.743350 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ca292-5764-4ed4-906b-96214963e117 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:38.817712 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6da00-24da-4edb-83c5-f62445c8d012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:38.818355 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6da00-24da-4edb-83c5-f62445c8d012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:38.818521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6da00-24da-4edb-83c5-f62445c8d012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:38.819005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6da00-24da-4edb-83c5-f62445c8d012 admin admin] PATH_INFO: `/v3/projects/1c3ed6ff48b44385acef77c00e631b6e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:38.822312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6da00-24da-4edb-83c5-f62445c8d012 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c3ed6ff48b44385acef77c00e631b6e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:38.822636 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31a9d4-54e7-44e4-b6fa-9d60c13e0a6f admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:38.824391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6da00-24da-4edb-83c5-f62445c8d012 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:38.829919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183ab1c-451d-405d-b6d1-9bb3d4e54b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:38.831133 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183ab1c-451d-405d-b6d1-9bb3d4e54b07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:38.831530 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183ab1c-451d-405d-b6d1-9bb3d4e54b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:38.832141 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183ab1c-451d-405d-b6d1-9bb3d4e54b07 admin admin] PATH_INFO: `/v3/projects/cc99c2c20a0346998ee35ae8aadb6e77` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:38.833436 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183ab1c-451d-405d-b6d1-9bb3d4e54b07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc99c2c20a0346998ee35ae8aadb6e77)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:38.836267 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183ab1c-451d-405d-b6d1-9bb3d4e54b07 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:38.936923 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b5a6da00-24da-4edb-83c5-f62445c8d012 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c3ed6ff48b44385acef77c00e631b6e', 'request_id': 'req-b5a6da00-24da-4edb-83c5-f62445c8d012'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:38.981634 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c183ab1c-451d-405d-b6d1-9bb3d4e54b07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc99c2c20a0346998ee35ae8aadb6e77', 'request_id': 'req-c183ab1c-451d-405d-b6d1-9bb3d4e54b07'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:38.988207 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c183ab1c-451d-405d-b6d1-9bb3d4e54b07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc99c2c20a0346998ee35ae8aadb6e77', 'request_id': 'req-c183ab1c-451d-405d-b6d1-9bb3d4e54b07'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:38.995170 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c183ab1c-451d-405d-b6d1-9bb3d4e54b07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc99c2c20a0346998ee35ae8aadb6e77', 'request_id': 'req-c183ab1c-451d-405d-b6d1-9bb3d4e54b07'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:39.056896 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b5a6da00-24da-4edb-83c5-f62445c8d012 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c3ed6ff48b44385acef77c00e631b6e', 'request_id': 'req-b5a6da00-24da-4edb-83c5-f62445c8d012'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:39.064780 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b5a6da00-24da-4edb-83c5-f62445c8d012 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c3ed6ff48b44385acef77c00e631b6e', 'request_id': 'req-b5a6da00-24da-4edb-83c5-f62445c8d012'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:39.077516 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b5a6da00-24da-4edb-83c5-f62445c8d012 admin admin] The token cache is being invalidate because project 1c3ed6ff48b44385acef77c00e631b6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:08:39.077865 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b5a6da00-24da-4edb-83c5-f62445c8d012 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:39.078691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b5a6da00-24da-4edb-83c5-f62445c8d012 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:39.080273 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1339/2657] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:08:38 2022] DELETE /identity/v3/projects/1c3ed6ff48b44385acef77c00e631b6e => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:39.085608 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287ec86-c689-41c9-890f-c0d06706c0d9 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:39.086349 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287ec86-c689-41c9-890f-c0d06706c0d9 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:39.086349 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287ec86-c689-41c9-890f-c0d06706c0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:39.107550 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-7287ec86-c689-41c9-890f-c0d06706c0d9 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:39.108752 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-7287ec86-c689-41c9-890f-c0d06706c0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:39.155450 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c183ab1c-451d-405d-b6d1-9bb3d4e54b07 admin admin] The token cache is being invalidate because project cc99c2c20a0346998ee35ae8aadb6e77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:08:39.156376 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c183ab1c-451d-405d-b6d1-9bb3d4e54b07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:39.157099 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c183ab1c-451d-405d-b6d1-9bb3d4e54b07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:39.159032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1319/2658] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:08:38 2022] DELETE /identity/v3/projects/cc99c2c20a0346998ee35ae8aadb6e77 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:39.162795 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183ab1c-451d-405d-b6d1-9bb3d4e54b07 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:39.164987 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c183ab1c-451d-405d-b6d1-9bb3d4e54b07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:39.172918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1340/2659] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:08:39.174705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:39.181473 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1d862-9ef3-4a08-bb20-a34fb190bbf0 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:39.181758 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1d862-9ef3-4a08-bb20-a34fb190bbf0 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:39.182027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1d862-9ef3-4a08-bb20-a34fb190bbf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:39.202822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-97b1d862-9ef3-4a08-bb20-a34fb190bbf0 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:39.204008 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-97b1d862-9ef3-4a08-bb20-a34fb190bbf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:39.213063 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1341/2660] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:08:39.217320 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b1d862-9ef3-4a08-bb20-a34fb190bbf0 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:39.226630 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b1d862-9ef3-4a08-bb20-a34fb190bbf0 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:39.231214 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f222b5-96d4-4137-90d2-dd4cfd4bcf18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:39.231787 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f222b5-96d4-4137-90d2-dd4cfd4bcf18 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:39.232056 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f222b5-96d4-4137-90d2-dd4cfd4bcf18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:39.232319 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f222b5-96d4-4137-90d2-dd4cfd4bcf18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:39.232768 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183ab1c-451d-405d-b6d1-9bb3d4e54b07 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:39.232940 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f222b5-96d4-4137-90d2-dd4cfd4bcf18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:08:39.233229 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f222b5-96d4-4137-90d2-dd4cfd4bcf18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:39.234816 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f222b5-96d4-4137-90d2-dd4cfd4bcf18 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:39.242772 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e740e-8612-46d5-8f0c-3c7e664a7d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:39.243555 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e740e-8612-46d5-8f0c-3c7e664a7d15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:39.243839 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e740e-8612-46d5-8f0c-3c7e664a7d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:39.244712 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e740e-8612-46d5-8f0c-3c7e664a7d15 admin admin] PATH_INFO: `/v3/users/5f818b3fa9864108902889acfdb36709` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:39.246142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e740e-8612-46d5-8f0c-3c7e664a7d15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f818b3fa9864108902889acfdb36709)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:39.247542 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1342/2661] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:08:39.248950 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e740e-8612-46d5-8f0c-3c7e664a7d15 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:39.251622 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f222b5-96d4-4137-90d2-dd4cfd4bcf18 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:39.256624 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:39.262529 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f222b5-96d4-4137-90d2-dd4cfd4bcf18 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:39.267286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0de01-15af-4344-ac63-1494f723e0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:39.267894 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0de01-15af-4344-ac63-1494f723e0b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:39.268164 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0de01-15af-4344-ac63-1494f723e0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:39.268424 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0de01-15af-4344-ac63-1494f723e0b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:39.269067 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0de01-15af-4344-ac63-1494f723e0b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:39.270993 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0de01-15af-4344-ac63-1494f723e0b9 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:39.399858 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1343/2662] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:08:39 2022] POST /identity/v3/projects => generated 383 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:08:39.404066 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0de01-15af-4344-ac63-1494f723e0b9 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:39.465176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0de01-15af-4344-ac63-1494f723e0b9 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:39.469866 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f206f39-bc5c-4d54-a7b3-4acd2a2bcf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:39.470457 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f206f39-bc5c-4d54-a7b3-4acd2a2bcf9c admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:39.470753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f206f39-bc5c-4d54-a7b3-4acd2a2bcf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:39.471022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f206f39-bc5c-4d54-a7b3-4acd2a2bcf9c admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:39.471670 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f206f39-bc5c-4d54-a7b3-4acd2a2bcf9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:39.473579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f206f39-bc5c-4d54-a7b3-4acd2a2bcf9c admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:39.487496 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ae9e740e-8612-46d5-8f0c-3c7e664a7d15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f818b3fa9864108902889acfdb36709'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:39.495262 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ae9e740e-8612-46d5-8f0c-3c7e664a7d15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f818b3fa9864108902889acfdb36709'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:39.509671 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ae9e740e-8612-46d5-8f0c-3c7e664a7d15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f818b3fa9864108902889acfdb36709'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:39.564856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1344/2663] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:08:39 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:08:39.568808 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f206f39-bc5c-4d54-a7b3-4acd2a2bcf9c admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:39.578449 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f206f39-bc5c-4d54-a7b3-4acd2a2bcf9c admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:39.583053 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bbb91c-8e61-488b-a716-340f4c05c983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:39.583615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bbb91c-8e61-488b-a716-340f4c05c983 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:39.583904 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bbb91c-8e61-488b-a716-340f4c05c983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:39.584152 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bbb91c-8e61-488b-a716-340f4c05c983 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:39.584769 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bbb91c-8e61-488b-a716-340f4c05c983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:39.586647 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bbb91c-8e61-488b-a716-340f4c05c983 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:39.587417 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ae9e740e-8612-46d5-8f0c-3c7e664a7d15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f818b3fa9864108902889acfdb36709'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:39.588136 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ae9e740e-8612-46d5-8f0c-3c7e664a7d15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f818b3fa9864108902889acfdb36709'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:39.595009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1320/2664] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:08:39 2022] DELETE /identity/v3/users/5f818b3fa9864108902889acfdb36709 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:08:39.603380 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1345/2665] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:08:39.613190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e740e-8612-46d5-8f0c-3c7e664a7d15 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:39.614717 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ae9e740e-8612-46d5-8f0c-3c7e664a7d15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:39.690016 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e740e-8612-46d5-8f0c-3c7e664a7d15 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:39.706292 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ec670-7b81-479a-9d83-51db5238dadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:39.706292 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ec670-7b81-479a-9d83-51db5238dadf admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:39.706292 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ec670-7b81-479a-9d83-51db5238dadf admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:39.706292 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ec670-7b81-479a-9d83-51db5238dadf admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:39.708532 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ec670-7b81-479a-9d83-51db5238dadf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:39.708532 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ec670-7b81-479a-9d83-51db5238dadf admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:39.713053 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1321/2666] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:08:39.717440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bbb91c-8e61-488b-a716-340f4c05c983 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:39.728433 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bbb91c-8e61-488b-a716-340f4c05c983 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:39.733299 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a119af-131a-4e8f-9508-5264f347c7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:39.733939 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a119af-131a-4e8f-9508-5264f347c7c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:39.734211 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a119af-131a-4e8f-9508-5264f347c7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:39.734483 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a119af-131a-4e8f-9508-5264f347c7c8 admin admin] PATH_INFO: `/v3/projects/261177a9929f4df4bff1d0c009b1b3e2/users/6ae921c806734b0a8218ea7453e8bd4e/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:39.751662 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a119af-131a-4e8f-9508-5264f347c7c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=261177a9929f4df4bff1d0c009b1b3e2, user_id=6ae921c806734b0a8218ea7453e8bd4e, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:39.754426 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a119af-131a-4e8f-9508-5264f347c7c8 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:39.983137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1346/2667] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:08:39 2022] PUT /identity/v3/projects/261177a9929f4df4bff1d0c009b1b3e2/users/6ae921c806734b0a8218ea7453e8bd4e/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:39.988196 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc2ee2-eaa6-4e85-8201-31a556670f50 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:39.988548 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc2ee2-eaa6-4e85-8201-31a556670f50 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:39.988909 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc2ee2-eaa6-4e85-8201-31a556670f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:40.012493 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-edbc2ee2-eaa6-4e85-8201-31a556670f50 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:40.013797 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-edbc2ee2-eaa6-4e85-8201-31a556670f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:40.032754 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:40.076068 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1322/2668] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:08:40.086641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a119af-131a-4e8f-9508-5264f347c7c8 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:40.088508 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e2a119af-131a-4e8f-9508-5264f347c7c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:40.153076 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a119af-131a-4e8f-9508-5264f347c7c8 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:40.156676 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b662f-fe28-4443-984e-4a6f3a918d95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:40.157280 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b662f-fe28-4443-984e-4a6f3a918d95 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:40.157561 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b662f-fe28-4443-984e-4a6f3a918d95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:40.157832 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b662f-fe28-4443-984e-4a6f3a918d95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:40.169649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b662f-fe28-4443-984e-4a6f3a918d95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:40.172101 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b662f-fe28-4443-984e-4a6f3a918d95 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:40.184909 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1347/2669] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:08:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:08:40.201501 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc2ee2-eaa6-4e85-8201-31a556670f50 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:40.203183 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-edbc2ee2-eaa6-4e85-8201-31a556670f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:40.222438 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc2ee2-eaa6-4e85-8201-31a556670f50 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:40.229025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb23433-e7a4-43e2-8f6d-46a8d4735d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:40.229658 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb23433-e7a4-43e2-8f6d-46a8d4735d95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:40.229936 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb23433-e7a4-43e2-8f6d-46a8d4735d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:40.230206 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb23433-e7a4-43e2-8f6d-46a8d4735d95 admin admin] PATH_INFO: `/v3/projects/3d8078fbb994454ba5f4c344ff647859` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:40.231421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb23433-e7a4-43e2-8f6d-46a8d4735d95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d8078fbb994454ba5f4c344ff647859)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:40.233367 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb23433-e7a4-43e2-8f6d-46a8d4735d95 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:40.349244 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ddb23433-e7a4-43e2-8f6d-46a8d4735d95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d8078fbb994454ba5f4c344ff647859', 'request_id': 'req-ddb23433-e7a4-43e2-8f6d-46a8d4735d95'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:40.358084 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ddb23433-e7a4-43e2-8f6d-46a8d4735d95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d8078fbb994454ba5f4c344ff647859', 'request_id': 'req-ddb23433-e7a4-43e2-8f6d-46a8d4735d95'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:40.364720 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ddb23433-e7a4-43e2-8f6d-46a8d4735d95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d8078fbb994454ba5f4c344ff647859', 'request_id': 'req-ddb23433-e7a4-43e2-8f6d-46a8d4735d95'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:40.438584 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ddb23433-e7a4-43e2-8f6d-46a8d4735d95 admin admin] The token cache is being invalidate because project 3d8078fbb994454ba5f4c344ff647859 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:08:40.443507 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ddb23433-e7a4-43e2-8f6d-46a8d4735d95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:40.443507 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ddb23433-e7a4-43e2-8f6d-46a8d4735d95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:40.443507 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1323/2670] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:08:40 2022] DELETE /identity/v3/projects/3d8078fbb994454ba5f4c344ff647859 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:40.445347 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b662f-fe28-4443-984e-4a6f3a918d95 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:40.455458 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b662f-fe28-4443-984e-4a6f3a918d95 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:40.508613 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb23433-e7a4-43e2-8f6d-46a8d4735d95 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:40.508769 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:40.510468 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ddb23433-e7a4-43e2-8f6d-46a8d4735d95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:40.538765 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5744d8-4bfb-4513-bfd6-2ca680e5acc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:40.539557 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5744d8-4bfb-4513-bfd6-2ca680e5acc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:40.539913 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5744d8-4bfb-4513-bfd6-2ca680e5acc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:40.540353 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5744d8-4bfb-4513-bfd6-2ca680e5acc3 admin admin] PATH_INFO: `/v3/users/1a0816887b69416389458fb6d0309213` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:40.541619 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5744d8-4bfb-4513-bfd6-2ca680e5acc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a0816887b69416389458fb6d0309213)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:40.545050 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5744d8-4bfb-4513-bfd6-2ca680e5acc3 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:40.580354 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb23433-e7a4-43e2-8f6d-46a8d4735d95 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:40.584060 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07360295-a237-41d7-bf81-3d1e39cd64fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:40.584719 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07360295-a237-41d7-bf81-3d1e39cd64fb service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:40.585567 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07360295-a237-41d7-bf81-3d1e39cd64fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:40.585923 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07360295-a237-41d7-bf81-3d1e39cd64fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:40.587740 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-07360295-a237-41d7-bf81-3d1e39cd64fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:40.647925 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:40.650041 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07360295-a237-41d7-bf81-3d1e39cd64fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:40.651523 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07360295-a237-41d7-bf81-3d1e39cd64fb service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:40.662053 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1324/2671] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:08:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:08:40.722343 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7d5744d8-4bfb-4513-bfd6-2ca680e5acc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a0816887b69416389458fb6d0309213'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:40.735820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7d5744d8-4bfb-4513-bfd6-2ca680e5acc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a0816887b69416389458fb6d0309213'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:40.743914 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7d5744d8-4bfb-4513-bfd6-2ca680e5acc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a0816887b69416389458fb6d0309213'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:40.744737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7d5744d8-4bfb-4513-bfd6-2ca680e5acc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a0816887b69416389458fb6d0309213'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:40.827573 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7d5744d8-4bfb-4513-bfd6-2ca680e5acc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a0816887b69416389458fb6d0309213'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:40.834141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1348/2672] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:08:40 2022] DELETE /identity/v3/users/1a0816887b69416389458fb6d0309213 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:41.379969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07360295-a237-41d7-bf81-3d1e39cd64fb service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:41.382913 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-07360295-a237-41d7-bf81-3d1e39cd64fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:41.396485 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07360295-a237-41d7-bf81-3d1e39cd64fb service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:41.498350 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3c6c2-cd87-4a56-839b-6925ace3563e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:41.502982 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3c6c2-cd87-4a56-839b-6925ace3563e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:41.502982 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3c6c2-cd87-4a56-839b-6925ace3563e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:41.502982 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3c6c2-cd87-4a56-839b-6925ace3563e admin admin] PATH_INFO: `/v3/projects/f1080d461e1f4ba48df112a1e3b1bcae` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:41.502982 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3c6c2-cd87-4a56-839b-6925ace3563e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1080d461e1f4ba48df112a1e3b1bcae)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:41.504112 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3c6c2-cd87-4a56-839b-6925ace3563e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:41.644080 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-82a3c6c2-cd87-4a56-839b-6925ace3563e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1080d461e1f4ba48df112a1e3b1bcae', 'request_id': 'req-82a3c6c2-cd87-4a56-839b-6925ace3563e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:41.652539 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-82a3c6c2-cd87-4a56-839b-6925ace3563e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1080d461e1f4ba48df112a1e3b1bcae', 'request_id': 'req-82a3c6c2-cd87-4a56-839b-6925ace3563e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:41.662002 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-82a3c6c2-cd87-4a56-839b-6925ace3563e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1080d461e1f4ba48df112a1e3b1bcae', 'request_id': 'req-82a3c6c2-cd87-4a56-839b-6925ace3563e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:41.730379 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-82a3c6c2-cd87-4a56-839b-6925ace3563e admin admin] The token cache is being invalidate because project f1080d461e1f4ba48df112a1e3b1bcae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:08:41.730796 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-82a3c6c2-cd87-4a56-839b-6925ace3563e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:41.731837 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-82a3c6c2-cd87-4a56-839b-6925ace3563e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:41.733336 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1325/2673] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:08:41 2022] DELETE /identity/v3/projects/f1080d461e1f4ba48df112a1e3b1bcae => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:41.739392 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165ff0f-78a1-4bb2-99b7-7638f8f8df91 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:41.739918 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165ff0f-78a1-4bb2-99b7-7638f8f8df91 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:41.740186 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165ff0f-78a1-4bb2-99b7-7638f8f8df91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:41.783847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-d165ff0f-78a1-4bb2-99b7-7638f8f8df91 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:41.785304 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-d165ff0f-78a1-4bb2-99b7-7638f8f8df91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:41.890407 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1349/2674] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:08:41.899475 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1cc66-079c-4ed0-b63b-465f9a2357fe None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:41.899763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1cc66-079c-4ed0-b63b-465f9a2357fe None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:41.900040 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1cc66-079c-4ed0-b63b-465f9a2357fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:41.929635 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c3f1cc66-079c-4ed0-b63b-465f9a2357fe None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:41.930935 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c3f1cc66-079c-4ed0-b63b-465f9a2357fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:41.939005 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1326/2675] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:08:41.943660 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165ff0f-78a1-4bb2-99b7-7638f8f8df91 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:41.954983 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165ff0f-78a1-4bb2-99b7-7638f8f8df91 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:41.960025 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b1b2d-0ca2-439e-8d1d-790eb62b12ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:41.961003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b1b2d-0ca2-439e-8d1d-790eb62b12ee admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:41.961377 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b1b2d-0ca2-439e-8d1d-790eb62b12ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:41.961759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b1b2d-0ca2-439e-8d1d-790eb62b12ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:41.962516 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b1b2d-0ca2-439e-8d1d-790eb62b12ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:08:41.967779 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b1b2d-0ca2-439e-8d1d-790eb62b12ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:41.977025 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b1b2d-0ca2-439e-8d1d-790eb62b12ee admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:42.007938 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1cc66-079c-4ed0-b63b-465f9a2357fe None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:42.021898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1350/2676] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:08:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:08:42.029718 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1cc66-079c-4ed0-b63b-465f9a2357fe None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:42.036099 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b811c-9334-46a0-9cf8-5564613f2813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:42.036828 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b811c-9334-46a0-9cf8-5564613f2813 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:42.038081 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b811c-9334-46a0-9cf8-5564613f2813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:42.039089 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b811c-9334-46a0-9cf8-5564613f2813 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:42.039423 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0b811c-9334-46a0-9cf8-5564613f2813 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:42.041468 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0b811c-9334-46a0-9cf8-5564613f2813 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:42.153975 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1327/2677] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:08:42 2022] POST /identity/v3/projects => generated 381 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:08:42.155508 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b1b2d-0ca2-439e-8d1d-790eb62b12ee admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:42.167489 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b1b2d-0ca2-439e-8d1d-790eb62b12ee admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:42.215413 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:42.231462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44cc6a1-ca95-4c1c-8ece-cd26276fa4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:42.232074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cc6a1-ca95-4c1c-8ece-cd26276fa4f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:42.232354 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cc6a1-ca95-4c1c-8ece-cd26276fa4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:42.232627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cc6a1-ca95-4c1c-8ece-cd26276fa4f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:42.233260 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44cc6a1-ca95-4c1c-8ece-cd26276fa4f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:42.235233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44cc6a1-ca95-4c1c-8ece-cd26276fa4f9 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:42.339819 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1351/2678] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:08:42 2022] POST /identity/v3/users => generated 291 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:08:42.340260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b811c-9334-46a0-9cf8-5564613f2813 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:42.355100 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b811c-9334-46a0-9cf8-5564613f2813 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:42.364609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376fd3e-2a8c-4f5f-a370-9150683df59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:42.365365 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376fd3e-2a8c-4f5f-a370-9150683df59d admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:42.365741 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376fd3e-2a8c-4f5f-a370-9150683df59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:42.366202 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376fd3e-2a8c-4f5f-a370-9150683df59d admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:42.367068 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376fd3e-2a8c-4f5f-a370-9150683df59d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:42.368917 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376fd3e-2a8c-4f5f-a370-9150683df59d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:42.389447 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44cc6a1-ca95-4c1c-8ece-cd26276fa4f9 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:42.393999 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1328/2679] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:08:42.402873 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44cc6a1-ca95-4c1c-8ece-cd26276fa4f9 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:42.418458 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d20d5e-8f4c-403c-8ebc-ff6f1641c10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:42.420629 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d20d5e-8f4c-403c-8ebc-ff6f1641c10a admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:42.420629 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d20d5e-8f4c-403c-8ebc-ff6f1641c10a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:42.420629 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d20d5e-8f4c-403c-8ebc-ff6f1641c10a admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:42.420629 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d20d5e-8f4c-403c-8ebc-ff6f1641c10a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:42.422687 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d20d5e-8f4c-403c-8ebc-ff6f1641c10a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:42.438013 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1352/2680] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:08:42.442296 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376fd3e-2a8c-4f5f-a370-9150683df59d admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:42.460969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376fd3e-2a8c-4f5f-a370-9150683df59d admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:42.467837 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11378d78-6faf-44c4-b767-52af0fe2664a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:42.468671 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11378d78-6faf-44c4-b767-52af0fe2664a admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:42.469016 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11378d78-6faf-44c4-b767-52af0fe2664a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:42.469359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11378d78-6faf-44c4-b767-52af0fe2664a admin admin] PATH_INFO: `/v3/projects/1e2ae9f944734d4da432caf60da0ea4a/users/47d187793fa94596bd9829fb6eab2d3d/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:42.493991 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11378d78-6faf-44c4-b767-52af0fe2664a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e2ae9f944734d4da432caf60da0ea4a, user_id=47d187793fa94596bd9829fb6eab2d3d, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:42.499509 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11378d78-6faf-44c4-b767-52af0fe2664a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:42.594699 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1329/2681] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:08:42 2022] PUT /identity/v3/projects/1e2ae9f944734d4da432caf60da0ea4a/users/47d187793fa94596bd9829fb6eab2d3d/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:42.609153 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bb114-9b5c-4f00-adc2-d4ee7d5d6d3a None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:42.621998 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bb114-9b5c-4f00-adc2-d4ee7d5d6d3a None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:42.621998 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bb114-9b5c-4f00-adc2-d4ee7d5d6d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:42.657079 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-dc3bb114-9b5c-4f00-adc2-d4ee7d5d6d3a None None] MFA Rules not processed for user `47d187793fa94596bd9829fb6eab2d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:42.658547 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-dc3bb114-9b5c-4f00-adc2-d4ee7d5d6d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:42.745323 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1353/2682] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:08:42.748697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11378d78-6faf-44c4-b767-52af0fe2664a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:42.790221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:42.849267 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11378d78-6faf-44c4-b767-52af0fe2664a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:42.855144 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a992a-e456-4e6a-9d39-443d833a7701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:42.855845 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a992a-e456-4e6a-9d39-443d833a7701 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:42.856356 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a992a-e456-4e6a-9d39-443d833a7701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:42.856734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a992a-e456-4e6a-9d39-443d833a7701 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:42.857526 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a992a-e456-4e6a-9d39-443d833a7701 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:42.859671 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a992a-e456-4e6a-9d39-443d833a7701 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:42.975487 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1330/2683] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:08:42 2022] POST /identity/v3/projects => generated 381 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:08:42.976398 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3bb114-9b5c-4f00-adc2-d4ee7d5d6d3a None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:42.990099 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3bb114-9b5c-4f00-adc2-d4ee7d5d6d3a None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:43.085217 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8f419-d4bd-4a15-9d57-c8b471d3052e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:43.085848 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8f419-d4bd-4a15-9d57-c8b471d3052e admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:43.086122 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8f419-d4bd-4a15-9d57-c8b471d3052e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:43.086393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8f419-d4bd-4a15-9d57-c8b471d3052e admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:43.087083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a8f419-d4bd-4a15-9d57-c8b471d3052e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:43.089064 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a8f419-d4bd-4a15-9d57-c8b471d3052e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:43.196024 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1354/2684] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:08:42 2022] POST /identity/v3/users => generated 297 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:08:43.202613 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a992a-e456-4e6a-9d39-443d833a7701 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:43.219872 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a992a-e456-4e6a-9d39-443d833a7701 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:43.227484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61eebe4-dcd6-488a-842e-28e1b4096a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:43.230409 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61eebe4-dcd6-488a-842e-28e1b4096a8e admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:43.230409 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61eebe4-dcd6-488a-842e-28e1b4096a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:43.230409 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61eebe4-dcd6-488a-842e-28e1b4096a8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:43.230975 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61eebe4-dcd6-488a-842e-28e1b4096a8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:43.232506 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61eebe4-dcd6-488a-842e-28e1b4096a8e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:43.248148 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1331/2685] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:08:43.252691 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8f419-d4bd-4a15-9d57-c8b471d3052e admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:43.271792 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8f419-d4bd-4a15-9d57-c8b471d3052e admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:43.282917 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36315970-99f0-4a55-b913-4be077b80bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:43.283729 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36315970-99f0-4a55-b913-4be077b80bd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:43.284149 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36315970-99f0-4a55-b913-4be077b80bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:43.284518 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36315970-99f0-4a55-b913-4be077b80bd1 admin admin] PATH_INFO: `/v3/projects/b4331552108c4cdd9fed9c1dcdaa0e9f/users/b95ce2d2937842a48bd6ec4385fccc0d/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:43.319071 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36315970-99f0-4a55-b913-4be077b80bd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4331552108c4cdd9fed9c1dcdaa0e9f, user_id=b95ce2d2937842a48bd6ec4385fccc0d, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:43.327190 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36315970-99f0-4a55-b913-4be077b80bd1 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:43.428724 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1355/2686] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:08:43 2022] PUT /identity/v3/projects/b4331552108c4cdd9fed9c1dcdaa0e9f/users/b95ce2d2937842a48bd6ec4385fccc0d/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:43.430678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61eebe4-dcd6-488a-842e-28e1b4096a8e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:43.458284 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61eebe4-dcd6-488a-842e-28e1b4096a8e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:43.550863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b3474-6522-4c34-9b04-5a922a72bf43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:43.551813 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b3474-6522-4c34-9b04-5a922a72bf43 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:43.552286 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b3474-6522-4c34-9b04-5a922a72bf43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:43.552632 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b3474-6522-4c34-9b04-5a922a72bf43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:43.553348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b3474-6522-4c34-9b04-5a922a72bf43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:43.556128 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b3474-6522-4c34-9b04-5a922a72bf43 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:43.574837 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1332/2687] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:08:43.579619 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36315970-99f0-4a55-b913-4be077b80bd1 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:43.596840 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36315970-99f0-4a55-b913-4be077b80bd1 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:43.608476 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e89a1a-e2ee-4510-907c-9136e2421b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:43.608476 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e89a1a-e2ee-4510-907c-9136e2421b27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:43.608476 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e89a1a-e2ee-4510-907c-9136e2421b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:43.608476 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e89a1a-e2ee-4510-907c-9136e2421b27 admin admin] PATH_INFO: `/v3/projects/b4331552108c4cdd9fed9c1dcdaa0e9f/users/b95ce2d2937842a48bd6ec4385fccc0d/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:43.613662 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e89a1a-e2ee-4510-907c-9136e2421b27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4331552108c4cdd9fed9c1dcdaa0e9f, user_id=b95ce2d2937842a48bd6ec4385fccc0d, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:43.617077 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e89a1a-e2ee-4510-907c-9136e2421b27 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:43.638632 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1356/2688] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:08:43 2022] PUT /identity/v3/projects/b4331552108c4cdd9fed9c1dcdaa0e9f/users/b95ce2d2937842a48bd6ec4385fccc0d/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:08:43.650906 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219b458-a1ab-47cf-a7b8-e7368ecc77c0 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:43.651190 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219b458-a1ab-47cf-a7b8-e7368ecc77c0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:43.651955 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219b458-a1ab-47cf-a7b8-e7368ecc77c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:43.684081 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5219b458-a1ab-47cf-a7b8-e7368ecc77c0 None None] MFA Rules not processed for user `b95ce2d2937842a48bd6ec4385fccc0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:43.685225 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5219b458-a1ab-47cf-a7b8-e7368ecc77c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:43.779277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1333/2689] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:43 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:08:43.792001 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e89a1a-e2ee-4510-907c-9136e2421b27 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:43.793627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e4e89a1a-e2ee-4510-907c-9136e2421b27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:43.802025 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:43.870118 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e89a1a-e2ee-4510-907c-9136e2421b27 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:43.876536 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d946912-4d97-4fcd-baf8-3da777667ce6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:43.882968 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d946912-4d97-4fcd-baf8-3da777667ce6 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:43.887398 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d946912-4d97-4fcd-baf8-3da777667ce6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:43.887554 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d946912-4d97-4fcd-baf8-3da777667ce6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:43.904790 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d946912-4d97-4fcd-baf8-3da777667ce6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:43.906859 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d946912-4d97-4fcd-baf8-3da777667ce6 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:43.936485 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1357/2690] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:08:43 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:08:44.714336 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219b458-a1ab-47cf-a7b8-e7368ecc77c0 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:44.738447 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:44.832035 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219b458-a1ab-47cf-a7b8-e7368ecc77c0 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:44.842925 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03251a32-afd3-4393-8d37-8a4728505583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:44.843875 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03251a32-afd3-4393-8d37-8a4728505583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:44.844162 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03251a32-afd3-4393-8d37-8a4728505583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:44.844560 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03251a32-afd3-4393-8d37-8a4728505583 admin admin] PATH_INFO: `/v3/users/47d187793fa94596bd9829fb6eab2d3d` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:44.845871 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03251a32-afd3-4393-8d37-8a4728505583 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47d187793fa94596bd9829fb6eab2d3d)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:44.848018 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03251a32-afd3-4393-8d37-8a4728505583 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:45.061787 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-03251a32-afd3-4393-8d37-8a4728505583 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47d187793fa94596bd9829fb6eab2d3d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:45.072809 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-03251a32-afd3-4393-8d37-8a4728505583 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47d187793fa94596bd9829fb6eab2d3d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:45.097076 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-03251a32-afd3-4393-8d37-8a4728505583 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47d187793fa94596bd9829fb6eab2d3d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:45.181505 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-03251a32-afd3-4393-8d37-8a4728505583 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47d187793fa94596bd9829fb6eab2d3d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:45.182167 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-03251a32-afd3-4393-8d37-8a4728505583 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47d187793fa94596bd9829fb6eab2d3d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:45.190740 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1334/2691] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:08:44 2022] DELETE /identity/v3/users/47d187793fa94596bd9829fb6eab2d3d => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:45.196021 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd346b5-9a32-4e6e-a82a-6aaed7002552 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:45.196997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd346b5-9a32-4e6e-a82a-6aaed7002552 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:45.197535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd346b5-9a32-4e6e-a82a-6aaed7002552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:45.221219 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-2dd346b5-9a32-4e6e-a82a-6aaed7002552 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:45.223920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2dd346b5-9a32-4e6e-a82a-6aaed7002552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:45.291273 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1358/2692] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:08:45.303038 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03251a32-afd3-4393-8d37-8a4728505583 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:45.304757 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-03251a32-afd3-4393-8d37-8a4728505583 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:45.375546 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03251a32-afd3-4393-8d37-8a4728505583 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:45.379636 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334467b-c8c8-4e6c-8ef2-042186730ebf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:45.380247 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334467b-c8c8-4e6c-8ef2-042186730ebf service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:45.380517 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334467b-c8c8-4e6c-8ef2-042186730ebf service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:45.380794 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334467b-c8c8-4e6c-8ef2-042186730ebf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:45.400748 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f334467b-c8c8-4e6c-8ef2-042186730ebf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:45.402290 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f334467b-c8c8-4e6c-8ef2-042186730ebf service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:45.415137 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1335/2693] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:08:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:08:45.831516 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd346b5-9a32-4e6e-a82a-6aaed7002552 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:45.833402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2dd346b5-9a32-4e6e-a82a-6aaed7002552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:45.851185 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd346b5-9a32-4e6e-a82a-6aaed7002552 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:45.856310 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c29f2-1f42-463b-80a4-3447e2b0f251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:45.857352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c29f2-1f42-463b-80a4-3447e2b0f251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:45.857497 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c29f2-1f42-463b-80a4-3447e2b0f251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:45.857775 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c29f2-1f42-463b-80a4-3447e2b0f251 admin admin] PATH_INFO: `/v3/projects/1e2ae9f944734d4da432caf60da0ea4a` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:45.861693 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c29f2-1f42-463b-80a4-3447e2b0f251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e2ae9f944734d4da432caf60da0ea4a)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:45.863657 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c29f2-1f42-463b-80a4-3447e2b0f251 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:46.005620 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7c0c29f2-1f42-463b-80a4-3447e2b0f251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e2ae9f944734d4da432caf60da0ea4a', 'request_id': 'req-7c0c29f2-1f42-463b-80a4-3447e2b0f251'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:46.113297 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7c0c29f2-1f42-463b-80a4-3447e2b0f251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e2ae9f944734d4da432caf60da0ea4a', 'request_id': 'req-7c0c29f2-1f42-463b-80a4-3447e2b0f251'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:46.123730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7c0c29f2-1f42-463b-80a4-3447e2b0f251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e2ae9f944734d4da432caf60da0ea4a', 'request_id': 'req-7c0c29f2-1f42-463b-80a4-3447e2b0f251'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:46.141417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7c0c29f2-1f42-463b-80a4-3447e2b0f251 admin admin] The token cache is being invalidate because project 1e2ae9f944734d4da432caf60da0ea4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:08:46.141578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7c0c29f2-1f42-463b-80a4-3447e2b0f251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:46.142459 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7c0c29f2-1f42-463b-80a4-3447e2b0f251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:46.144539 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1359/2694] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:08:45 2022] DELETE /identity/v3/projects/1e2ae9f944734d4da432caf60da0ea4a => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:46.148699 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334467b-c8c8-4e6c-8ef2-042186730ebf service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:46.159032 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334467b-c8c8-4e6c-8ef2-042186730ebf service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:46.219053 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:46.242086 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e7d20-a819-4955-993b-3d7720cad86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:46.243137 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e7d20-a819-4955-993b-3d7720cad86e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:46.244474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e7d20-a819-4955-993b-3d7720cad86e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:46.244474 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e7d20-a819-4955-993b-3d7720cad86e admin admin] PATH_INFO: `/v3/users/b95ce2d2937842a48bd6ec4385fccc0d` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:46.246090 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2e7d20-a819-4955-993b-3d7720cad86e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b95ce2d2937842a48bd6ec4385fccc0d)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:46.248554 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2e7d20-a819-4955-993b-3d7720cad86e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:46.260292 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872e68f-84d8-4faf-a55c-7c41cdc80912 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:46.260837 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872e68f-84d8-4faf-a55c-7c41cdc80912 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:46.261106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872e68f-84d8-4faf-a55c-7c41cdc80912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:46.296380 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-e872e68f-84d8-4faf-a55c-7c41cdc80912 None None] MFA Rules not processed for user `6ae921c806734b0a8218ea7453e8bd4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:46.302818 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e872e68f-84d8-4faf-a55c-7c41cdc80912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:46.325091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:46.375261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1360/2695] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:08:46.390017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e872e68f-84d8-4faf-a55c-7c41cdc80912 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:46.391476 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-e872e68f-84d8-4faf-a55c-7c41cdc80912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:46.449363 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3c2e7d20-a819-4955-993b-3d7720cad86e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b95ce2d2937842a48bd6ec4385fccc0d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:46.457923 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3c2e7d20-a819-4955-993b-3d7720cad86e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b95ce2d2937842a48bd6ec4385fccc0d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:46.471469 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3c2e7d20-a819-4955-993b-3d7720cad86e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b95ce2d2937842a48bd6ec4385fccc0d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:46.475076 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e872e68f-84d8-4faf-a55c-7c41cdc80912 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:46.481135 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa7b03-b4fa-41dc-8c9f-acdc93404f4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:46.481752 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa7b03-b4fa-41dc-8c9f-acdc93404f4e service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:46.482048 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa7b03-b4fa-41dc-8c9f-acdc93404f4e service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:46.482289 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa7b03-b4fa-41dc-8c9f-acdc93404f4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:46.492196 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa7b03-b4fa-41dc-8c9f-acdc93404f4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:46.493821 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa7b03-b4fa-41dc-8c9f-acdc93404f4e service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:46.503403 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1361/2696] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:08:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:08:46.542752 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3c2e7d20-a819-4955-993b-3d7720cad86e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b95ce2d2937842a48bd6ec4385fccc0d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:46.543472 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-3c2e7d20-a819-4955-993b-3d7720cad86e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b95ce2d2937842a48bd6ec4385fccc0d'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:46.550312 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1336/2697] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:08:46 2022] DELETE /identity/v3/users/b95ce2d2937842a48bd6ec4385fccc0d => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:46.972198 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa7b03-b4fa-41dc-8c9f-acdc93404f4e service nova] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:46.974119 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-19fa7b03-b4fa-41dc-8c9f-acdc93404f4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:46.992378 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa7b03-b4fa-41dc-8c9f-acdc93404f4e service nova] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:47.054973 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa2429-a33d-43d4-b942-3c96156da6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:47.055618 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa2429-a33d-43d4-b942-3c96156da6f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:47.055896 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa2429-a33d-43d4-b942-3c96156da6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:47.056168 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa2429-a33d-43d4-b942-3c96156da6f3 admin admin] PATH_INFO: `/v3/projects/b4331552108c4cdd9fed9c1dcdaa0e9f` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:47.057306 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa2429-a33d-43d4-b942-3c96156da6f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4331552108c4cdd9fed9c1dcdaa0e9f)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:47.059251 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa2429-a33d-43d4-b942-3c96156da6f3 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:47.209602 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:47.211554 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f0aa2429-a33d-43d4-b942-3c96156da6f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4331552108c4cdd9fed9c1dcdaa0e9f', 'request_id': 'req-f0aa2429-a33d-43d4-b942-3c96156da6f3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:47.278430 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f0aa2429-a33d-43d4-b942-3c96156da6f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4331552108c4cdd9fed9c1dcdaa0e9f', 'request_id': 'req-f0aa2429-a33d-43d4-b942-3c96156da6f3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:47.285037 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f0aa2429-a33d-43d4-b942-3c96156da6f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4331552108c4cdd9fed9c1dcdaa0e9f', 'request_id': 'req-f0aa2429-a33d-43d4-b942-3c96156da6f3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:47.296572 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f0aa2429-a33d-43d4-b942-3c96156da6f3 admin admin] The token cache is being invalidate because project b4331552108c4cdd9fed9c1dcdaa0e9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:08:47.296860 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f0aa2429-a33d-43d4-b942-3c96156da6f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:47.297512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f0aa2429-a33d-43d4-b942-3c96156da6f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:08:47.302732 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1362/2698] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:08:46 2022] DELETE /identity/v3/projects/b4331552108c4cdd9fed9c1dcdaa0e9f => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:47.305595 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d4bd9-5563-435a-b5fc-5cfe2d86e947 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:47.306068 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d4bd9-5563-435a-b5fc-5cfe2d86e947 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:47.306182 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d4bd9-5563-435a-b5fc-5cfe2d86e947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:47.335521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-805d4bd9-5563-435a-b5fc-5cfe2d86e947 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:47.336759 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-805d4bd9-5563-435a-b5fc-5cfe2d86e947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:47.401891 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1337/2699] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:08:47.410897 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63d388-daf6-4a32-923f-c4c8f06b2ff1 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:47.411189 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63d388-daf6-4a32-923f-c4c8f06b2ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:47.411459 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63d388-daf6-4a32-923f-c4c8f06b2ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:47.434478 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-dc63d388-daf6-4a32-923f-c4c8f06b2ff1 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:47.436478 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-dc63d388-daf6-4a32-923f-c4c8f06b2ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:47.447152 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1363/2700] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:08:47.450015 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d4bd9-5563-435a-b5fc-5cfe2d86e947 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:47.460518 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d4bd9-5563-435a-b5fc-5cfe2d86e947 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:47.465796 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a34bc-ecae-48bd-a6a1-c595b47547a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:47.466550 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a34bc-ecae-48bd-a6a1-c595b47547a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:47.469389 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a34bc-ecae-48bd-a6a1-c595b47547a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:47.469389 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a34bc-ecae-48bd-a6a1-c595b47547a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:47.469389 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a34bc-ecae-48bd-a6a1-c595b47547a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:08:47.469389 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a34bc-ecae-48bd-a6a1-c595b47547a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:47.470613 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a34bc-ecae-48bd-a6a1-c595b47547a7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:47.484727 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1338/2701] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:08:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:08:47.489132 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63d388-daf6-4a32-923f-c4c8f06b2ff1 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:47.499346 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63d388-daf6-4a32-923f-c4c8f06b2ff1 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:47.503984 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82663ec-2cf9-4379-b47f-0ae8534e0602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:47.504582 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82663ec-2cf9-4379-b47f-0ae8534e0602 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:47.504858 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82663ec-2cf9-4379-b47f-0ae8534e0602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:47.505127 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82663ec-2cf9-4379-b47f-0ae8534e0602 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:47.505786 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82663ec-2cf9-4379-b47f-0ae8534e0602 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:47.507756 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82663ec-2cf9-4379-b47f-0ae8534e0602 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:47.594773 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1364/2702] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:08:47 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:08:47.596143 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a34bc-ecae-48bd-a6a1-c595b47547a7 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:47.612078 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a34bc-ecae-48bd-a6a1-c595b47547a7 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:47.682356 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:47.703820 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142c8c1-46a4-40d1-90ba-c57256cca74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:47.709695 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142c8c1-46a4-40d1-90ba-c57256cca74f admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:47.709695 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142c8c1-46a4-40d1-90ba-c57256cca74f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:47.709695 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142c8c1-46a4-40d1-90ba-c57256cca74f admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:47.709695 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6142c8c1-46a4-40d1-90ba-c57256cca74f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:47.709695 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6142c8c1-46a4-40d1-90ba-c57256cca74f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:47.823784 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1339/2703] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:08:47 2022] POST /identity/v3/users => generated 289 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:08:47.827963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82663ec-2cf9-4379-b47f-0ae8534e0602 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:47.839428 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82663ec-2cf9-4379-b47f-0ae8534e0602 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:47.845479 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74946262-d716-4a17-bf80-d9c19cc90710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:47.850119 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74946262-d716-4a17-bf80-d9c19cc90710 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:47.850119 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74946262-d716-4a17-bf80-d9c19cc90710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:47.850119 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74946262-d716-4a17-bf80-d9c19cc90710 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:47.850119 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74946262-d716-4a17-bf80-d9c19cc90710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:47.850119 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74946262-d716-4a17-bf80-d9c19cc90710 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:47.864327 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1365/2704] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:08:47.864999 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142c8c1-46a4-40d1-90ba-c57256cca74f admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:47.876956 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142c8c1-46a4-40d1-90ba-c57256cca74f admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:47.883938 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82b841-65cd-4c94-ab78-53f3d890a95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:47.884152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82b841-65cd-4c94-ab78-53f3d890a95d admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:47.884436 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82b841-65cd-4c94-ab78-53f3d890a95d admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:47.884781 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82b841-65cd-4c94-ab78-53f3d890a95d admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:47.886001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a82b841-65cd-4c94-ab78-53f3d890a95d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:47.887488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a82b841-65cd-4c94-ab78-53f3d890a95d admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:47.905587 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1340/2705] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:08:47.909381 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74946262-d716-4a17-bf80-d9c19cc90710 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:47.920367 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74946262-d716-4a17-bf80-d9c19cc90710 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:47.925233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb652f-16b8-4090-a56a-e3ec35a79958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:47.925898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb652f-16b8-4090-a56a-e3ec35a79958 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:47.926177 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb652f-16b8-4090-a56a-e3ec35a79958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:47.926448 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb652f-16b8-4090-a56a-e3ec35a79958 admin admin] PATH_INFO: `/v3/projects/af7d161c2b204ba8aa801bc0ce7ed487/users/6069108acf564ac2a5a7383b9e20352f/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:47.943696 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deb652f-16b8-4090-a56a-e3ec35a79958 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af7d161c2b204ba8aa801bc0ce7ed487, user_id=6069108acf564ac2a5a7383b9e20352f, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:47.947102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deb652f-16b8-4090-a56a-e3ec35a79958 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:48.037663 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1366/2706] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:08:47 2022] PUT /identity/v3/projects/af7d161c2b204ba8aa801bc0ce7ed487/users/6069108acf564ac2a5a7383b9e20352f/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:08:48.049304 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6f6a2-0ed5-4058-bc73-562018208528 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:48.049582 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6f6a2-0ed5-4058-bc73-562018208528 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:48.049856 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6f6a2-0ed5-4058-bc73-562018208528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:48.078846 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-2ef6f6a2-0ed5-4058-bc73-562018208528 None None] MFA Rules not processed for user `6069108acf564ac2a5a7383b9e20352f`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:48.080076 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2ef6f6a2-0ed5-4058-bc73-562018208528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:48.145785 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1341/2707] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:08:48.157921 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb652f-16b8-4090-a56a-e3ec35a79958 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:08:48.159541 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5deb652f-16b8-4090-a56a-e3ec35a79958 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:48.216110 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb652f-16b8-4090-a56a-e3ec35a79958 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:08:48.220021 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094e274-cd92-4cdd-aa51-5da3d21c22e1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cb18199dc8b42589184042556e98983', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:08:48.220622 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094e274-cd92-4cdd-aa51-5da3d21c22e1 service glance] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:48.220899 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094e274-cd92-4cdd-aa51-5da3d21c22e1 service glance] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:48.221181 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094e274-cd92-4cdd-aa51-5da3d21c22e1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:48.230207 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1094e274-cd92-4cdd-aa51-5da3d21c22e1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:08:48.231790 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1094e274-cd92-4cdd-aa51-5da3d21c22e1 service glance] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:08:48.240981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1367/2708] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:08:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:08:49.029285 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1dc564-3e27-4eaf-b147-b513904cb146 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:08:49.029793 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1dc564-3e27-4eaf-b147-b513904cb146 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:08:49.029904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1dc564-3e27-4eaf-b147-b513904cb146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:08:49.062633 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-8e1dc564-3e27-4eaf-b147-b513904cb146 None None] MFA Rules not processed for user `6ae921c806734b0a8218ea7453e8bd4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:08:49.065974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8e1dc564-3e27-4eaf-b147-b513904cb146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:08:49.136838 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1342/2709] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:08:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:08:58.236771 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=60,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:08:59.127960 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=70,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:09:02.461766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc8bcf-2e88-4b92-9147-53b42813b242 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:02.462265 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc8bcf-2e88-4b92-9147-53b42813b242 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:02.462364 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc8bcf-2e88-4b92-9147-53b42813b242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:02.482046 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-1fbc8bcf-2e88-4b92-9147-53b42813b242 None None] MFA Rules not processed for user `b0a879085a194b0687e1cdcbafd48033`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:09:02.483274 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1fbc8bcf-2e88-4b92-9147-53b42813b242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:02.545985 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1368/2710] 10.222.0.17 () {58 vars in 967 bytes} [Sun Sep 11 00:09:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:09:02.682704 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1dc564-3e27-4eaf-b147-b513904cb146 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:02.684158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8e1dc564-3e27-4eaf-b147-b513904cb146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:02.752244 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1dc564-3e27-4eaf-b147-b513904cb146 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:02.757354 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dcb386-25b4-47c8-aaf5-4934260af448 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '564e65585e9543e286b6041af870cf41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:02.757946 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dcb386-25b4-47c8-aaf5-4934260af448 service swift] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:02.758222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dcb386-25b4-47c8-aaf5-4934260af448 service swift] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:02.758490 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dcb386-25b4-47c8-aaf5-4934260af448 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:02.766855 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dcb386-25b4-47c8-aaf5-4934260af448 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:02.768650 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dcb386-25b4-47c8-aaf5-4934260af448 service swift] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:02.777807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1343/2711] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Sep 11 00:09:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 00:09:06.787495 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc8bcf-2e88-4b92-9147-53b42813b242 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:06.876009 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc8bcf-2e88-4b92-9147-53b42813b242 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:06.880722 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268c83d-c732-4d96-a02b-12e658b576f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:06.881363 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268c83d-c732-4d96-a02b-12e658b576f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:06.882327 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268c83d-c732-4d96-a02b-12e658b576f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:06.882751 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268c83d-c732-4d96-a02b-12e658b576f4 admin admin] PATH_INFO: `/v3/users/6069108acf564ac2a5a7383b9e20352f` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:06.886607 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2268c83d-c732-4d96-a02b-12e658b576f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6069108acf564ac2a5a7383b9e20352f)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:06.886607 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2268c83d-c732-4d96-a02b-12e658b576f4 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:07.048518 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2268c83d-c732-4d96-a02b-12e658b576f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6069108acf564ac2a5a7383b9e20352f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:07.065154 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2268c83d-c732-4d96-a02b-12e658b576f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6069108acf564ac2a5a7383b9e20352f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:07.072844 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2268c83d-c732-4d96-a02b-12e658b576f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6069108acf564ac2a5a7383b9e20352f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:07.073515 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2268c83d-c732-4d96-a02b-12e658b576f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6069108acf564ac2a5a7383b9e20352f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:07.118923 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2268c83d-c732-4d96-a02b-12e658b576f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6069108acf564ac2a5a7383b9e20352f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:07.133226 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1369/2712] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:09:06 2022] DELETE /identity/v3/users/6069108acf564ac2a5a7383b9e20352f => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:09:07.134907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840ed36-15b5-42e5-bc78-3fcdf03ef74a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:07.135362 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840ed36-15b5-42e5-bc78-3fcdf03ef74a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:07.135480 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840ed36-15b5-42e5-bc78-3fcdf03ef74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:07.155540 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b840ed36-15b5-42e5-bc78-3fcdf03ef74a None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:09:07.156767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b840ed36-15b5-42e5-bc78-3fcdf03ef74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:07.222726 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1344/2713] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:09:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:09:07.233382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268c83d-c732-4d96-a02b-12e658b576f4 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:07.234856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2268c83d-c732-4d96-a02b-12e658b576f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:07.255617 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:09:07.295254 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268c83d-c732-4d96-a02b-12e658b576f4 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:07.301863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923cf94-adc4-4a26-af20-6fd0c5494cd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:07.302637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923cf94-adc4-4a26-af20-6fd0c5494cd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:07.303039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923cf94-adc4-4a26-af20-6fd0c5494cd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:07.303941 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923cf94-adc4-4a26-af20-6fd0c5494cd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:07.321314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1923cf94-adc4-4a26-af20-6fd0c5494cd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:07.323550 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1923cf94-adc4-4a26-af20-6fd0c5494cd3 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:07.336175 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1370/2714] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:09:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:09:07.512525 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840ed36-15b5-42e5-bc78-3fcdf03ef74a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:07.514572 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b840ed36-15b5-42e5-bc78-3fcdf03ef74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:07.619294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840ed36-15b5-42e5-bc78-3fcdf03ef74a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:07.624616 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ef708-73d1-4d8e-9555-1a9a236bec46 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cb1d85a338e40f18e61887a1becb906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:07.625260 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ef708-73d1-4d8e-9555-1a9a236bec46 service placement] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:07.625492 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ef708-73d1-4d8e-9555-1a9a236bec46 service placement] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:07.625763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ef708-73d1-4d8e-9555-1a9a236bec46 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:07.627527 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a82ef708-73d1-4d8e-9555-1a9a236bec46 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:07.640943 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ef708-73d1-4d8e-9555-1a9a236bec46 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:07.642636 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ef708-73d1-4d8e-9555-1a9a236bec46 service placement] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:07.649873 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:09:07.656479 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1345/2715] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:09:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:09:08.052193 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923cf94-adc4-4a26-af20-6fd0c5494cd3 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:08.054989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1923cf94-adc4-4a26-af20-6fd0c5494cd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:08.072686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923cf94-adc4-4a26-af20-6fd0c5494cd3 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:08.078352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa06bc-5309-45ab-9411-326844b8573f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:08.079436 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa06bc-5309-45ab-9411-326844b8573f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:08.079901 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa06bc-5309-45ab-9411-326844b8573f admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:08.080249 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa06bc-5309-45ab-9411-326844b8573f admin admin] PATH_INFO: `/v3/projects/af7d161c2b204ba8aa801bc0ce7ed487` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:08.081516 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa06bc-5309-45ab-9411-326844b8573f admin admin] RBAC: Authorizing `identity:delete_project(project_id=af7d161c2b204ba8aa801bc0ce7ed487)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:08.083532 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa06bc-5309-45ab-9411-326844b8573f admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:08.183901 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2ffa06bc-5309-45ab-9411-326844b8573f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af7d161c2b204ba8aa801bc0ce7ed487', 'request_id': 'req-2ffa06bc-5309-45ab-9411-326844b8573f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:08.254937 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2ffa06bc-5309-45ab-9411-326844b8573f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af7d161c2b204ba8aa801bc0ce7ed487', 'request_id': 'req-2ffa06bc-5309-45ab-9411-326844b8573f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:08.261641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2ffa06bc-5309-45ab-9411-326844b8573f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af7d161c2b204ba8aa801bc0ce7ed487', 'request_id': 'req-2ffa06bc-5309-45ab-9411-326844b8573f'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:08.273408 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2ffa06bc-5309-45ab-9411-326844b8573f admin admin] The token cache is being invalidate because project af7d161c2b204ba8aa801bc0ce7ed487 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:09:08.273686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2ffa06bc-5309-45ab-9411-326844b8573f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:08.274329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2ffa06bc-5309-45ab-9411-326844b8573f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:08.275826 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1371/2716] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:09:08 2022] DELETE /identity/v3/projects/af7d161c2b204ba8aa801bc0ce7ed487 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:09:08.280703 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99b280-0df6-4cf6-b006-6b630ff4322a None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:08.281255 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99b280-0df6-4cf6-b006-6b630ff4322a None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:08.281255 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99b280-0df6-4cf6-b006-6b630ff4322a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:08.303374 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ef99b280-0df6-4cf6-b006-6b630ff4322a None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:09:08.305538 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ef99b280-0df6-4cf6-b006-6b630ff4322a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:08.373860 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1346/2717] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:09:08.382108 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8d0bc-2840-4b79-ae7e-e5ea7155fff6 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:08.382471 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8d0bc-2840-4b79-ae7e-e5ea7155fff6 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:08.382846 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8d0bc-2840-4b79-ae7e-e5ea7155fff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:08.403645 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-6ba8d0bc-2840-4b79-ae7e-e5ea7155fff6 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:09:08.405137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6ba8d0bc-2840-4b79-ae7e-e5ea7155fff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:08.413982 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1372/2718] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:09:08.418312 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef99b280-0df6-4cf6-b006-6b630ff4322a None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:08.427899 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef99b280-0df6-4cf6-b006-6b630ff4322a None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:08.433062 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3a2ea-f259-409a-bf1f-e5fcc3592a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:08.433651 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3a2ea-f259-409a-bf1f-e5fcc3592a41 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:08.433927 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3a2ea-f259-409a-bf1f-e5fcc3592a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:08.434192 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3a2ea-f259-409a-bf1f-e5fcc3592a41 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:08.434855 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a3a2ea-f259-409a-bf1f-e5fcc3592a41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:09:08.435168 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a3a2ea-f259-409a-bf1f-e5fcc3592a41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:08.436748 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a3a2ea-f259-409a-bf1f-e5fcc3592a41 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:08.448216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1347/2719] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:09:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:09:08.452920 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8d0bc-2840-4b79-ae7e-e5ea7155fff6 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:08.462347 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8d0bc-2840-4b79-ae7e-e5ea7155fff6 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:08.467021 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a91f3-e9e5-4dbf-b4d2-f0a963f504b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:08.467641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a91f3-e9e5-4dbf-b4d2-f0a963f504b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:08.467917 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a91f3-e9e5-4dbf-b4d2-f0a963f504b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:08.468185 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a91f3-e9e5-4dbf-b4d2-f0a963f504b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:08.468807 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a91f3-e9e5-4dbf-b4d2-f0a963f504b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:08.470716 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a91f3-e9e5-4dbf-b4d2-f0a963f504b0 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:08.582883 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1373/2720] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:09:08 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:09:08.587890 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3a2ea-f259-409a-bf1f-e5fcc3592a41 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:08.597848 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3a2ea-f259-409a-bf1f-e5fcc3592a41 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:08.663616 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ce74b-b842-4d46-9610-fd3cbdacfaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:08.664222 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ce74b-b842-4d46-9610-fd3cbdacfaa6 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:08.664499 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ce74b-b842-4d46-9610-fd3cbdacfaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:08.664882 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ce74b-b842-4d46-9610-fd3cbdacfaa6 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:08.665593 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0ce74b-b842-4d46-9610-fd3cbdacfaa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:08.667731 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0ce74b-b842-4d46-9610-fd3cbdacfaa6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:08.757910 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1348/2721] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:09:08 2022] POST /identity/v3/users => generated 287 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:09:08.759326 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a91f3-e9e5-4dbf-b4d2-f0a963f504b0 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:08.769827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a91f3-e9e5-4dbf-b4d2-f0a963f504b0 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:08.774778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a99d50-ccba-4e16-8a56-7371d37bb36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:08.775385 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a99d50-ccba-4e16-8a56-7371d37bb36f admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:08.775659 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a99d50-ccba-4e16-8a56-7371d37bb36f admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:08.775927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a99d50-ccba-4e16-8a56-7371d37bb36f admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:08.776621 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a99d50-ccba-4e16-8a56-7371d37bb36f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:08.778214 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a99d50-ccba-4e16-8a56-7371d37bb36f admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:08.802290 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1374/2722] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:09:08.803162 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ce74b-b842-4d46-9610-fd3cbdacfaa6 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:08.818753 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ce74b-b842-4d46-9610-fd3cbdacfaa6 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:08.824588 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae3dd8-f851-4417-93b3-e1f7318005ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:08.825203 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae3dd8-f851-4417-93b3-e1f7318005ea admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:08.825478 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae3dd8-f851-4417-93b3-e1f7318005ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:08.825744 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae3dd8-f851-4417-93b3-e1f7318005ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:08.826381 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae3dd8-f851-4417-93b3-e1f7318005ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:08.828264 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae3dd8-f851-4417-93b3-e1f7318005ea admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:08.851947 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1349/2723] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:09:08.853433 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a99d50-ccba-4e16-8a56-7371d37bb36f admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:08.863948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a99d50-ccba-4e16-8a56-7371d37bb36f admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:08.869453 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3bfd80-b79a-4d09-8230-be634302faa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:08.870123 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3bfd80-b79a-4d09-8230-be634302faa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:08.870395 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3bfd80-b79a-4d09-8230-be634302faa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:08.870684 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3bfd80-b79a-4d09-8230-be634302faa8 admin admin] PATH_INFO: `/v3/projects/e7928fd61ace426180ed42483ee48994/users/435e06748e1a43139b1df8e7ff62d805/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:08.891720 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3bfd80-b79a-4d09-8230-be634302faa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7928fd61ace426180ed42483ee48994, user_id=435e06748e1a43139b1df8e7ff62d805, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:08.896284 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3bfd80-b79a-4d09-8230-be634302faa8 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:08.971670 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1375/2724] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:09:08 2022] PUT /identity/v3/projects/e7928fd61ace426180ed42483ee48994/users/435e06748e1a43139b1df8e7ff62d805/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:09:09.045308 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c386cd-fafc-4aac-980a-a540536f0a04 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:09.045588 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c386cd-fafc-4aac-980a-a540536f0a04 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:09.045867 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c386cd-fafc-4aac-980a-a540536f0a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:09.078490 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c9c386cd-fafc-4aac-980a-a540536f0a04 None None] MFA Rules not processed for user `435e06748e1a43139b1df8e7ff62d805`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:09:09.079777 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c9c386cd-fafc-4aac-980a-a540536f0a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:09.136483 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:09:09.144803 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1350/2725] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:09:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:09:09.155972 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3bfd80-b79a-4d09-8230-be634302faa8 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:09.157666 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6b3bfd80-b79a-4d09-8230-be634302faa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:09.211534 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:09:09.225268 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3bfd80-b79a-4d09-8230-be634302faa8 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:09.229335 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c66eed-cb0d-4dc8-aa1d-8c76147fa923 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:09.229948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c66eed-cb0d-4dc8-aa1d-8c76147fa923 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:09.230316 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c66eed-cb0d-4dc8-aa1d-8c76147fa923 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:09.230588 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c66eed-cb0d-4dc8-aa1d-8c76147fa923 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:09.239948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c66eed-cb0d-4dc8-aa1d-8c76147fa923 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:09.241591 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c66eed-cb0d-4dc8-aa1d-8c76147fa923 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:09.251627 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1376/2726] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:09:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:09:11.057501 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf7bfd-1fc8-464c-abba-3680b0c3a740 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:11.058108 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf7bfd-1fc8-464c-abba-3680b0c3a740 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:11.058108 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf7bfd-1fc8-464c-abba-3680b0c3a740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:11.079090 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5adf7bfd-1fc8-464c-abba-3680b0c3a740 None None] MFA Rules not processed for user `435e06748e1a43139b1df8e7ff62d805`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:09:11.080309 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5adf7bfd-1fc8-464c-abba-3680b0c3a740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:11.090715 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1351/2727] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:09:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:09:11.463946 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c66eed-cb0d-4dc8-aa1d-8c76147fa923 service cinder] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:11.465924 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-49c66eed-cb0d-4dc8-aa1d-8c76147fa923 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:11.533679 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c66eed-cb0d-4dc8-aa1d-8c76147fa923 service cinder] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:11.538644 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5deb671-1bf3-45c9-ac9c-e9550489cc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:11.539299 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5deb671-1bf3-45c9-ac9c-e9550489cc54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:11.539574 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5deb671-1bf3-45c9-ac9c-e9550489cc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:11.539886 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5deb671-1bf3-45c9-ac9c-e9550489cc54 admin admin] PATH_INFO: `/v3/users/fd106d45a1ce4aad9c85c35d4f53af92` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:11.540952 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5deb671-1bf3-45c9-ac9c-e9550489cc54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd106d45a1ce4aad9c85c35d4f53af92)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:11.542919 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5deb671-1bf3-45c9-ac9c-e9550489cc54 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:11.744083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b5deb671-1bf3-45c9-ac9c-e9550489cc54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd106d45a1ce4aad9c85c35d4f53af92'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:11.758193 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b5deb671-1bf3-45c9-ac9c-e9550489cc54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd106d45a1ce4aad9c85c35d4f53af92'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:11.765344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b5deb671-1bf3-45c9-ac9c-e9550489cc54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd106d45a1ce4aad9c85c35d4f53af92'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:11.766040 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b5deb671-1bf3-45c9-ac9c-e9550489cc54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd106d45a1ce4aad9c85c35d4f53af92'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:11.849326 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-b5deb671-1bf3-45c9-ac9c-e9550489cc54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd106d45a1ce4aad9c85c35d4f53af92'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:11.855883 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1377/2728] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:09:11 2022] DELETE /identity/v3/users/fd106d45a1ce4aad9c85c35d4f53af92 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:09:12.115185 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf7bfd-1fc8-464c-abba-3680b0c3a740 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:12.126714 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf7bfd-1fc8-464c-abba-3680b0c3a740 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:12.185863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8ab26-14dc-4cbb-a65f-eaefc2a6dcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:12.186512 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8ab26-14dc-4cbb-a65f-eaefc2a6dcb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:12.187632 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8ab26-14dc-4cbb-a65f-eaefc2a6dcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:12.187998 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8ab26-14dc-4cbb-a65f-eaefc2a6dcb4 admin admin] PATH_INFO: `/v3/projects/82b6b05c598d44ee8b8748b6bab1f9e5` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:12.189236 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd8ab26-14dc-4cbb-a65f-eaefc2a6dcb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82b6b05c598d44ee8b8748b6bab1f9e5)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:12.191297 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd8ab26-14dc-4cbb-a65f-eaefc2a6dcb4 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:12.309342 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9dd8ab26-14dc-4cbb-a65f-eaefc2a6dcb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82b6b05c598d44ee8b8748b6bab1f9e5', 'request_id': 'req-9dd8ab26-14dc-4cbb-a65f-eaefc2a6dcb4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:12.315253 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9dd8ab26-14dc-4cbb-a65f-eaefc2a6dcb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82b6b05c598d44ee8b8748b6bab1f9e5', 'request_id': 'req-9dd8ab26-14dc-4cbb-a65f-eaefc2a6dcb4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:12.321855 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9dd8ab26-14dc-4cbb-a65f-eaefc2a6dcb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82b6b05c598d44ee8b8748b6bab1f9e5', 'request_id': 'req-9dd8ab26-14dc-4cbb-a65f-eaefc2a6dcb4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:12.386980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9dd8ab26-14dc-4cbb-a65f-eaefc2a6dcb4 admin admin] The token cache is being invalidate because project 82b6b05c598d44ee8b8748b6bab1f9e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:09:12.388397 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9dd8ab26-14dc-4cbb-a65f-eaefc2a6dcb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:12.388397 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9dd8ab26-14dc-4cbb-a65f-eaefc2a6dcb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:12.389537 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1352/2729] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:09:12 2022] DELETE /identity/v3/projects/82b6b05c598d44ee8b8748b6bab1f9e5 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:09:12.393931 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5deb671-1bf3-45c9-ac9c-e9550489cc54 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:12.395460 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b5deb671-1bf3-45c9-ac9c-e9550489cc54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:12.408673 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5deb671-1bf3-45c9-ac9c-e9550489cc54 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:12.465295 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1887a63-f022-4825-abbb-ce5e9f1c2d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:12.465966 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1887a63-f022-4825-abbb-ce5e9f1c2d01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:12.466166 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1887a63-f022-4825-abbb-ce5e9f1c2d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:12.466435 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1887a63-f022-4825-abbb-ce5e9f1c2d01 admin admin] PATH_INFO: `/v3/users/19fcb43f28984cd393dc9d9e28ac7c64` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:12.467587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1887a63-f022-4825-abbb-ce5e9f1c2d01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19fcb43f28984cd393dc9d9e28ac7c64)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:12.469523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1887a63-f022-4825-abbb-ce5e9f1c2d01 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:12.471806 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:09:12.675924 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d1887a63-f022-4825-abbb-ce5e9f1c2d01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19fcb43f28984cd393dc9d9e28ac7c64'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:12.688860 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d1887a63-f022-4825-abbb-ce5e9f1c2d01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19fcb43f28984cd393dc9d9e28ac7c64'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:12.696317 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d1887a63-f022-4825-abbb-ce5e9f1c2d01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19fcb43f28984cd393dc9d9e28ac7c64'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:12.696998 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d1887a63-f022-4825-abbb-ce5e9f1c2d01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19fcb43f28984cd393dc9d9e28ac7c64'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:12.753134 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-d1887a63-f022-4825-abbb-ce5e9f1c2d01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19fcb43f28984cd393dc9d9e28ac7c64'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:12.759509 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1378/2730] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:09:12 2022] DELETE /identity/v3/users/19fcb43f28984cd393dc9d9e28ac7c64 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:09:13.040110 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8ab26-14dc-4cbb-a65f-eaefc2a6dcb4 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:13.041527 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9dd8ab26-14dc-4cbb-a65f-eaefc2a6dcb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:13.054817 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8ab26-14dc-4cbb-a65f-eaefc2a6dcb4 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:13.118333 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe70cc-ee4d-4d42-bd95-eb046926a3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:13.118999 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe70cc-ee4d-4d42-bd95-eb046926a3f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:13.119279 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe70cc-ee4d-4d42-bd95-eb046926a3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:13.119551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe70cc-ee4d-4d42-bd95-eb046926a3f2 admin admin] PATH_INFO: `/v3/projects/cc0ef4d1c1a746e3a2467f2d852553f8` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:13.120787 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe70cc-ee4d-4d42-bd95-eb046926a3f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc0ef4d1c1a746e3a2467f2d852553f8)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:13.122721 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe70cc-ee4d-4d42-bd95-eb046926a3f2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:13.148444 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:09:13.221881 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d1fe70cc-ee4d-4d42-bd95-eb046926a3f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc0ef4d1c1a746e3a2467f2d852553f8', 'request_id': 'req-d1fe70cc-ee4d-4d42-bd95-eb046926a3f2'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:13.228021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d1fe70cc-ee4d-4d42-bd95-eb046926a3f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc0ef4d1c1a746e3a2467f2d852553f8', 'request_id': 'req-d1fe70cc-ee4d-4d42-bd95-eb046926a3f2'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:13.233777 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d1fe70cc-ee4d-4d42-bd95-eb046926a3f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc0ef4d1c1a746e3a2467f2d852553f8', 'request_id': 'req-d1fe70cc-ee4d-4d42-bd95-eb046926a3f2'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:13.295239 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d1fe70cc-ee4d-4d42-bd95-eb046926a3f2 admin admin] The token cache is being invalidate because project cc0ef4d1c1a746e3a2467f2d852553f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:09:13.295462 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d1fe70cc-ee4d-4d42-bd95-eb046926a3f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:13.296172 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d1fe70cc-ee4d-4d42-bd95-eb046926a3f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:13.297709 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1353/2731] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:09:13 2022] DELETE /identity/v3/projects/cc0ef4d1c1a746e3a2467f2d852553f8 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:09:13.303009 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc2aec-90c2-4135-bf09-450af0cc0f76 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:13.303932 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc2aec-90c2-4135-bf09-450af0cc0f76 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:13.304373 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc2aec-90c2-4135-bf09-450af0cc0f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:13.325667 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-0fdc2aec-90c2-4135-bf09-450af0cc0f76 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:09:13.327162 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-0fdc2aec-90c2-4135-bf09-450af0cc0f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:13.387659 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1379/2732] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:09:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:09:13.396128 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818f2db-fa7a-447b-8d0d-2593201b61cb None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:13.396414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818f2db-fa7a-447b-8d0d-2593201b61cb None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:13.396715 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818f2db-fa7a-447b-8d0d-2593201b61cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:13.418544 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5818f2db-fa7a-447b-8d0d-2593201b61cb None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:09:13.419916 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5818f2db-fa7a-447b-8d0d-2593201b61cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:13.428046 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1354/2733] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:09:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:09:13.433202 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc2aec-90c2-4135-bf09-450af0cc0f76 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:13.442887 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc2aec-90c2-4135-bf09-450af0cc0f76 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:13.448387 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd26837-3c3a-4a3f-a149-3b792145091a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:13.448988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd26837-3c3a-4a3f-a149-3b792145091a admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:13.449411 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd26837-3c3a-4a3f-a149-3b792145091a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:13.449788 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd26837-3c3a-4a3f-a149-3b792145091a admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:13.450433 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd26837-3c3a-4a3f-a149-3b792145091a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:09:13.450776 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd26837-3c3a-4a3f-a149-3b792145091a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:13.452634 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd26837-3c3a-4a3f-a149-3b792145091a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:13.464949 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1380/2734] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:09:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:09:13.469230 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818f2db-fa7a-447b-8d0d-2593201b61cb None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:13.479317 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818f2db-fa7a-447b-8d0d-2593201b61cb None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:13.484397 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a884d38-1090-4754-b64b-5a71b9286936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:13.484529 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a884d38-1090-4754-b64b-5a71b9286936 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:13.484756 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a884d38-1090-4754-b64b-5a71b9286936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:13.485023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a884d38-1090-4754-b64b-5a71b9286936 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:13.485649 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a884d38-1090-4754-b64b-5a71b9286936 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:13.487612 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a884d38-1090-4754-b64b-5a71b9286936 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:13.547963 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1355/2735] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:09:13 2022] POST /identity/v3/projects => generated 379 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:09:13.552200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd26837-3c3a-4a3f-a149-3b792145091a admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:13.562356 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd26837-3c3a-4a3f-a149-3b792145091a admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:13.618336 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff207e4-1a07-47a1-853d-685582843a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:13.618969 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff207e4-1a07-47a1-853d-685582843a8f admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:13.619249 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff207e4-1a07-47a1-853d-685582843a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:13.619520 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff207e4-1a07-47a1-853d-685582843a8f admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:13.620160 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff207e4-1a07-47a1-853d-685582843a8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:13.622050 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff207e4-1a07-47a1-853d-685582843a8f admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:13.684425 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:09:13.690159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1381/2736] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:09:13 2022] POST /identity/v3/users => generated 290 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:09:13.694436 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a884d38-1090-4754-b64b-5a71b9286936 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:13.705454 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a884d38-1090-4754-b64b-5a71b9286936 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:13.710345 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b302bd-cf5d-4bd1-b339-1d3ac33da018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:13.710970 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b302bd-cf5d-4bd1-b339-1d3ac33da018 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:13.711301 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b302bd-cf5d-4bd1-b339-1d3ac33da018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:13.711574 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b302bd-cf5d-4bd1-b339-1d3ac33da018 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:13.712562 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b302bd-cf5d-4bd1-b339-1d3ac33da018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:13.714235 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b302bd-cf5d-4bd1-b339-1d3ac33da018 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:13.724249 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1356/2737] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:09:13.728734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff207e4-1a07-47a1-853d-685582843a8f admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:13.739283 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff207e4-1a07-47a1-853d-685582843a8f admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:13.745536 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0138c9a-dd47-4222-8f5f-258a5677f973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:13.746613 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0138c9a-dd47-4222-8f5f-258a5677f973 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:13.746928 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0138c9a-dd47-4222-8f5f-258a5677f973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:13.747205 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0138c9a-dd47-4222-8f5f-258a5677f973 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:13.748016 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0138c9a-dd47-4222-8f5f-258a5677f973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:13.749619 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0138c9a-dd47-4222-8f5f-258a5677f973 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:13.761708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1382/2738] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:09:13.765865 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b302bd-cf5d-4bd1-b339-1d3ac33da018 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:13.776484 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b302bd-cf5d-4bd1-b339-1d3ac33da018 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:13.783497 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4cda1-a6b1-472f-94d9-1284da8f1d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:13.784785 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4cda1-a6b1-472f-94d9-1284da8f1d54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:13.785229 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4cda1-a6b1-472f-94d9-1284da8f1d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:13.785754 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4cda1-a6b1-472f-94d9-1284da8f1d54 admin admin] PATH_INFO: `/v3/projects/a819c981f44e4d21ab8b8289d16c19fb/users/aa4f534ee1ab4e639e99b9e4ebbe8363/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:13.803082 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4cda1-a6b1-472f-94d9-1284da8f1d54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a819c981f44e4d21ab8b8289d16c19fb, user_id=aa4f534ee1ab4e639e99b9e4ebbe8363, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:13.806693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4cda1-a6b1-472f-94d9-1284da8f1d54 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:13.879512 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1357/2739] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:09:13 2022] PUT /identity/v3/projects/a819c981f44e4d21ab8b8289d16c19fb/users/aa4f534ee1ab4e639e99b9e4ebbe8363/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:09:13.885255 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7e663-b40b-4506-819f-c3ce7200f431 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:13.885623 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7e663-b40b-4506-819f-c3ce7200f431 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:13.885989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7e663-b40b-4506-819f-c3ce7200f431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:13.905511 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-4ae7e663-b40b-4506-819f-c3ce7200f431 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:09:13.906778 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4ae7e663-b40b-4506-819f-c3ce7200f431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:13.967977 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1383/2740] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:09:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:09:13.983808 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4cda1-a6b1-472f-94d9-1284da8f1d54 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:13.985567 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e8b4cda1-a6b1-472f-94d9-1284da8f1d54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:14.047118 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4cda1-a6b1-472f-94d9-1284da8f1d54 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:14.051463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5b9a9-5888-463f-a274-2569d9d23e06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:14.053982 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5b9a9-5888-463f-a274-2569d9d23e06 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:14.053982 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5b9a9-5888-463f-a274-2569d9d23e06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:14.053982 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5b9a9-5888-463f-a274-2569d9d23e06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:14.062865 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd5b9a9-5888-463f-a274-2569d9d23e06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:14.064348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd5b9a9-5888-463f-a274-2569d9d23e06 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:14.076321 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1358/2741] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:09:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:09:19.596600 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fce3e-e11f-4fe9-8f86-bf0d7451f626 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:19.597107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fce3e-e11f-4fe9-8f86-bf0d7451f626 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:19.597107 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fce3e-e11f-4fe9-8f86-bf0d7451f626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:19.631082 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-a94fce3e-e11f-4fe9-8f86-bf0d7451f626 None None] MFA Rules not processed for user `aa4f534ee1ab4e639e99b9e4ebbe8363`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:09:19.633052 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a94fce3e-e11f-4fe9-8f86-bf0d7451f626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:19.702095 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1384/2742] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:09:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:09:19.706627 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5b9a9-5888-463f-a274-2569d9d23e06 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:19.733152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5b9a9-5888-463f-a274-2569d9d23e06 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:19.738807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02659e6c-8df5-4c0a-b83c-da2a708d87dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:19.739974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02659e6c-8df5-4c0a-b83c-da2a708d87dc admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:19.740256 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02659e6c-8df5-4c0a-b83c-da2a708d87dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:19.740530 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02659e6c-8df5-4c0a-b83c-da2a708d87dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:19.741766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02659e6c-8df5-4c0a-b83c-da2a708d87dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:19.743734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02659e6c-8df5-4c0a-b83c-da2a708d87dc admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:19.875441 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1359/2743] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:09:19 2022] POST /identity/v3/projects => generated 377 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:09:19.879730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94fce3e-e11f-4fe9-8f86-bf0d7451f626 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:19.893910 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94fce3e-e11f-4fe9-8f86-bf0d7451f626 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:19.963456 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6dcb45-f5e0-4d9a-9734-cbdef2c3993d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:19.964068 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6dcb45-f5e0-4d9a-9734-cbdef2c3993d admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:19.964346 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6dcb45-f5e0-4d9a-9734-cbdef2c3993d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:19.964756 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6dcb45-f5e0-4d9a-9734-cbdef2c3993d admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:19.965462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6dcb45-f5e0-4d9a-9734-cbdef2c3993d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:19.967739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6dcb45-f5e0-4d9a-9734-cbdef2c3993d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:20.068031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1385/2744] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:09:19 2022] POST /identity/v3/users => generated 295 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:09:20.072671 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02659e6c-8df5-4c0a-b83c-da2a708d87dc admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:20.086520 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02659e6c-8df5-4c0a-b83c-da2a708d87dc admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:20.091054 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4714da-4ce0-460e-8d37-5910c64bb2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:20.092187 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4714da-4ce0-460e-8d37-5910c64bb2b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:20.092187 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4714da-4ce0-460e-8d37-5910c64bb2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:20.092419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4714da-4ce0-460e-8d37-5910c64bb2b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:20.093025 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4714da-4ce0-460e-8d37-5910c64bb2b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:20.094637 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4714da-4ce0-460e-8d37-5910c64bb2b2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:20.110063 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1360/2745] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:09:20.112286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6dcb45-f5e0-4d9a-9734-cbdef2c3993d admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:20.123884 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6dcb45-f5e0-4d9a-9734-cbdef2c3993d admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:20.129549 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630817f9-27cb-465a-90d0-0c01fa77321a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:20.130375 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630817f9-27cb-465a-90d0-0c01fa77321a admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:20.130692 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630817f9-27cb-465a-90d0-0c01fa77321a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:20.131087 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630817f9-27cb-465a-90d0-0c01fa77321a admin admin] PATH_INFO: `/v3/projects/1fc676ca5ade41519650a1048e1eb544/users/73592f21d8d44b0dba097094351e37bb/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:20.135406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:09:20.151019 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630817f9-27cb-465a-90d0-0c01fa77321a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fc676ca5ade41519650a1048e1eb544, user_id=73592f21d8d44b0dba097094351e37bb, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:20.153669 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630817f9-27cb-465a-90d0-0c01fa77321a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:20.250612 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1386/2746] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:09:20 2022] PUT /identity/v3/projects/1fc676ca5ade41519650a1048e1eb544/users/73592f21d8d44b0dba097094351e37bb/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:09:20.254732 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4714da-4ce0-460e-8d37-5910c64bb2b2 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:20.265028 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4714da-4ce0-460e-8d37-5910c64bb2b2 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:20.277104 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:09:20.322751 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bfbf41-7174-48e9-8171-db9f00181698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:20.323367 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bfbf41-7174-48e9-8171-db9f00181698 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:20.323840 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bfbf41-7174-48e9-8171-db9f00181698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:20.324039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bfbf41-7174-48e9-8171-db9f00181698 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:20.324691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bfbf41-7174-48e9-8171-db9f00181698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:20.326295 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bfbf41-7174-48e9-8171-db9f00181698 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:20.337155 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1361/2747] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:09:20.341447 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630817f9-27cb-465a-90d0-0c01fa77321a admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:20.352455 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630817f9-27cb-465a-90d0-0c01fa77321a admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:20.357176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5f241-7797-48b3-9fe9-99ba70f81158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:20.357844 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5f241-7797-48b3-9fe9-99ba70f81158 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:20.358215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5f241-7797-48b3-9fe9-99ba70f81158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:20.358531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5f241-7797-48b3-9fe9-99ba70f81158 admin admin] PATH_INFO: `/v3/projects/1fc676ca5ade41519650a1048e1eb544/users/73592f21d8d44b0dba097094351e37bb/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:20.360708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5f241-7797-48b3-9fe9-99ba70f81158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fc676ca5ade41519650a1048e1eb544, user_id=73592f21d8d44b0dba097094351e37bb, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:20.363636 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5f241-7797-48b3-9fe9-99ba70f81158 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:20.375598 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1387/2748] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:09:20 2022] PUT /identity/v3/projects/1fc676ca5ade41519650a1048e1eb544/users/73592f21d8d44b0dba097094351e37bb/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:09:26.532372 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bcc87b-91d9-4e84-94c9-f04e6e083937 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:26.532898 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bcc87b-91d9-4e84-94c9-f04e6e083937 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:26.532898 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bcc87b-91d9-4e84-94c9-f04e6e083937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:26.564930 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-48bcc87b-91d9-4e84-94c9-f04e6e083937 None None] MFA Rules not processed for user `73592f21d8d44b0dba097094351e37bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:09:26.566346 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-48bcc87b-91d9-4e84-94c9-f04e6e083937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:26.645427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1362/2749] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:09:26 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:09:26.659955 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5f241-7797-48b3-9fe9-99ba70f81158 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:26.661597 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5cd5f241-7797-48b3-9fe9-99ba70f81158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:26.760227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5f241-7797-48b3-9fe9-99ba70f81158 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:26.765713 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90d537-7188-4014-b155-902c88a2ac8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:26.766313 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90d537-7188-4014-b155-902c88a2ac8a service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:26.766590 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90d537-7188-4014-b155-902c88a2ac8a service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:26.766891 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90d537-7188-4014-b155-902c88a2ac8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:26.824373 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd90d537-7188-4014-b155-902c88a2ac8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:26.826009 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd90d537-7188-4014-b155-902c88a2ac8a service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:26.836518 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1388/2750] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:09:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:09:30.065001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc8ac7-f82d-4d75-ae06-a1d5cf01bb9d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:30.065592 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc8ac7-f82d-4d75-ae06-a1d5cf01bb9d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:30.065592 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc8ac7-f82d-4d75-ae06-a1d5cf01bb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:30.086671 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-57bc8ac7-f82d-4d75-ae06-a1d5cf01bb9d None None] MFA Rules not processed for user `b0a879085a194b0687e1cdcbafd48033`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:09:30.088150 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-57bc8ac7-f82d-4d75-ae06-a1d5cf01bb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:30.144719 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1363/2751] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Sep 11 00:09:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:09:30.157794 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90d537-7188-4014-b155-902c88a2ac8a service nova] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:30.159223 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-cd90d537-7188-4014-b155-902c88a2ac8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:30.209357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:09:30.230054 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90d537-7188-4014-b155-902c88a2ac8a service nova] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:30.235008 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ac3b0-c744-4540-86fb-86d3d02a2a3f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '564e65585e9543e286b6041af870cf41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:30.236207 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ac3b0-c744-4540-86fb-86d3d02a2a3f service swift] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:30.236207 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ac3b0-c744-4540-86fb-86d3d02a2a3f service swift] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:30.236207 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ac3b0-c744-4540-86fb-86d3d02a2a3f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:30.245343 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ac3b0-c744-4540-86fb-86d3d02a2a3f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:30.247234 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ac3b0-c744-4540-86fb-86d3d02a2a3f service swift] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:30.255528 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1389/2752] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Sep 11 00:09:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 00:09:31.262339 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5da29-f2c7-406f-b188-9641792e1baa None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:31.262888 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5da29-f2c7-406f-b188-9641792e1baa None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:31.262985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5da29-f2c7-406f-b188-9641792e1baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:31.285501 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-15f5da29-f2c7-406f-b188-9641792e1baa None None] MFA Rules not processed for user `b0a879085a194b0687e1cdcbafd48033`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:09:31.286737 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-15f5da29-f2c7-406f-b188-9641792e1baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:31.297390 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1364/2753] 10.222.0.17 () {58 vars in 967 bytes} [Sun Sep 11 00:09:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:09:31.308502 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ac3b0-c744-4540-86fb-86d3d02a2a3f service swift] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:31.319604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ac3b0-c744-4540-86fb-86d3d02a2a3f service swift] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:31.324612 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4816b-e7bb-46a1-97c9-1a3ee852c3da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '564e65585e9543e286b6041af870cf41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:31.325197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4816b-e7bb-46a1-97c9-1a3ee852c3da service swift] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:31.325470 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4816b-e7bb-46a1-97c9-1a3ee852c3da service swift] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:31.325735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4816b-e7bb-46a1-97c9-1a3ee852c3da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:31.334374 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b4816b-e7bb-46a1-97c9-1a3ee852c3da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:31.336347 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b4816b-e7bb-46a1-97c9-1a3ee852c3da service swift] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:31.345462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1390/2754] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Sep 11 00:09:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 00:09:31.943656 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a2dea-dd76-4cf6-bcad-2144b9897af6 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:31.944127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a2dea-dd76-4cf6-bcad-2144b9897af6 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:31.944454 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a2dea-dd76-4cf6-bcad-2144b9897af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:31.971719 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-903a2dea-dd76-4cf6-bcad-2144b9897af6 None None] MFA Rules not processed for user `b0a879085a194b0687e1cdcbafd48033`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:09:31.972921 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-903a2dea-dd76-4cf6-bcad-2144b9897af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:31.981339 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1365/2755] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Sep 11 00:09:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:09:31.993577 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4816b-e7bb-46a1-97c9-1a3ee852c3da service swift] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:32.004233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4816b-e7bb-46a1-97c9-1a3ee852c3da service swift] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:32.009711 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ca614-e7d7-4135-a7f4-39021ca00b3c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '564e65585e9543e286b6041af870cf41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:32.010300 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ca614-e7d7-4135-a7f4-39021ca00b3c service swift] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:32.010576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ca614-e7d7-4135-a7f4-39021ca00b3c service swift] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:32.010883 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ca614-e7d7-4135-a7f4-39021ca00b3c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:32.018901 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ca614-e7d7-4135-a7f4-39021ca00b3c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:32.020437 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ca614-e7d7-4135-a7f4-39021ca00b3c service swift] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:32.029545 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1391/2756] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Sep 11 00:09:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 00:09:32.594877 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c4cf5-c1cc-4b2e-980b-f743cbdb80fb None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:32.595366 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c4cf5-c1cc-4b2e-980b-f743cbdb80fb None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:32.595464 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c4cf5-c1cc-4b2e-980b-f743cbdb80fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:32.617955 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-2a4c4cf5-c1cc-4b2e-980b-f743cbdb80fb None None] MFA Rules not processed for user `b0a879085a194b0687e1cdcbafd48033`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:09:32.619197 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2a4c4cf5-c1cc-4b2e-980b-f743cbdb80fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:32.626443 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1366/2757] 10.222.0.17 () {58 vars in 967 bytes} [Sun Sep 11 00:09:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:09:32.639432 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ca614-e7d7-4135-a7f4-39021ca00b3c service swift] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:32.651000 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ca614-e7d7-4135-a7f4-39021ca00b3c service swift] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:32.656866 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa44ffa-9847-4496-a14d-7252493ba177 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '564e65585e9543e286b6041af870cf41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:32.657476 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa44ffa-9847-4496-a14d-7252493ba177 service swift] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:32.657760 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa44ffa-9847-4496-a14d-7252493ba177 service swift] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:32.658032 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa44ffa-9847-4496-a14d-7252493ba177 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:32.667200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa44ffa-9847-4496-a14d-7252493ba177 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:32.668767 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa44ffa-9847-4496-a14d-7252493ba177 service swift] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:32.676698 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1392/2758] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Sep 11 00:09:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 00:09:35.281720 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c4cf5-c1cc-4b2e-980b-f743cbdb80fb None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:35.283132 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-2a4c4cf5-c1cc-4b2e-980b-f743cbdb80fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:35.313486 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=100 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:09:35.356249 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c4cf5-c1cc-4b2e-980b-f743cbdb80fb None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:35.361011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cfd1d-a0e5-4d28-b495-a9faf2a372c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:35.361630 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cfd1d-a0e5-4d28-b495-a9faf2a372c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:35.361905 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cfd1d-a0e5-4d28-b495-a9faf2a372c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:35.362173 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cfd1d-a0e5-4d28-b495-a9faf2a372c3 admin admin] PATH_INFO: `/v3/users/435e06748e1a43139b1df8e7ff62d805` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:35.363303 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5cfd1d-a0e5-4d28-b495-a9faf2a372c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=435e06748e1a43139b1df8e7ff62d805)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:35.365219 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5cfd1d-a0e5-4d28-b495-a9faf2a372c3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:35.527691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4f5cfd1d-a0e5-4d28-b495-a9faf2a372c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '435e06748e1a43139b1df8e7ff62d805'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:35.534703 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4f5cfd1d-a0e5-4d28-b495-a9faf2a372c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '435e06748e1a43139b1df8e7ff62d805'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:35.547467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4f5cfd1d-a0e5-4d28-b495-a9faf2a372c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '435e06748e1a43139b1df8e7ff62d805'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:35.633905 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4f5cfd1d-a0e5-4d28-b495-a9faf2a372c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '435e06748e1a43139b1df8e7ff62d805'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:35.634483 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4f5cfd1d-a0e5-4d28-b495-a9faf2a372c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '435e06748e1a43139b1df8e7ff62d805'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:35.640889 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1367/2759] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:09:35 2022] DELETE /identity/v3/users/435e06748e1a43139b1df8e7ff62d805 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:09:35.645532 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4280af-b1d4-4271-a1cc-57196c467e06 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:35.646021 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4280af-b1d4-4271-a1cc-57196c467e06 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:35.646117 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4280af-b1d4-4271-a1cc-57196c467e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:35.665998 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-ba4280af-b1d4-4271-a1cc-57196c467e06 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:09:35.666826 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ba4280af-b1d4-4271-a1cc-57196c467e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:35.727827 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1393/2760] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:09:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:09:35.738808 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cfd1d-a0e5-4d28-b495-a9faf2a372c3 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:35.740215 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4f5cfd1d-a0e5-4d28-b495-a9faf2a372c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:35.803119 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cfd1d-a0e5-4d28-b495-a9faf2a372c3 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:35.807695 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56cc38-e898-4675-b77f-4e5adf688f26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:35.808312 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56cc38-e898-4675-b77f-4e5adf688f26 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:35.808589 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56cc38-e898-4675-b77f-4e5adf688f26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:35.808854 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56cc38-e898-4675-b77f-4e5adf688f26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:35.824671 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56cc38-e898-4675-b77f-4e5adf688f26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:35.824671 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56cc38-e898-4675-b77f-4e5adf688f26 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:35.830443 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1368/2761] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:09:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:09:36.091800 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4280af-b1d4-4271-a1cc-57196c467e06 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:36.093245 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ba4280af-b1d4-4271-a1cc-57196c467e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:36.106215 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4280af-b1d4-4271-a1cc-57196c467e06 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:36.111466 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaccc44-ca67-49f5-8619-d9eb36ad8f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:36.112121 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaccc44-ca67-49f5-8619-d9eb36ad8f25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:36.112388 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaccc44-ca67-49f5-8619-d9eb36ad8f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:36.112652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaccc44-ca67-49f5-8619-d9eb36ad8f25 admin admin] PATH_INFO: `/v3/projects/e7928fd61ace426180ed42483ee48994` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:36.113748 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaccc44-ca67-49f5-8619-d9eb36ad8f25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7928fd61ace426180ed42483ee48994)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:36.115670 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaccc44-ca67-49f5-8619-d9eb36ad8f25 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:36.218200 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7aaccc44-ca67-49f5-8619-d9eb36ad8f25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7928fd61ace426180ed42483ee48994', 'request_id': 'req-7aaccc44-ca67-49f5-8619-d9eb36ad8f25'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:36.284375 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7aaccc44-ca67-49f5-8619-d9eb36ad8f25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7928fd61ace426180ed42483ee48994', 'request_id': 'req-7aaccc44-ca67-49f5-8619-d9eb36ad8f25'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:36.290632 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7aaccc44-ca67-49f5-8619-d9eb36ad8f25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7928fd61ace426180ed42483ee48994', 'request_id': 'req-7aaccc44-ca67-49f5-8619-d9eb36ad8f25'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:36.303653 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7aaccc44-ca67-49f5-8619-d9eb36ad8f25 admin admin] The token cache is being invalidate because project e7928fd61ace426180ed42483ee48994 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:09:36.304206 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7aaccc44-ca67-49f5-8619-d9eb36ad8f25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:36.305167 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7aaccc44-ca67-49f5-8619-d9eb36ad8f25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:36.306647 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1394/2762] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:09:36 2022] DELETE /identity/v3/projects/e7928fd61ace426180ed42483ee48994 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:09:36.311791 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e8be1-fa10-491d-85e8-406a631e9d2e None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:36.312314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e8be1-fa10-491d-85e8-406a631e9d2e None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:36.312314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e8be1-fa10-491d-85e8-406a631e9d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:36.334537 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-770e8be1-fa10-491d-85e8-406a631e9d2e None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:09:36.337127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-770e8be1-fa10-491d-85e8-406a631e9d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:36.381527 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:09:36.401550 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1369/2763] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:09:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:09:36.409620 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec680421-ee39-4105-a8d3-746eface129c None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:36.414432 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec680421-ee39-4105-a8d3-746eface129c None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:36.414432 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec680421-ee39-4105-a8d3-746eface129c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:36.430995 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-ec680421-ee39-4105-a8d3-746eface129c None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:09:36.432207 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ec680421-ee39-4105-a8d3-746eface129c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:36.440460 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1395/2764] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:09:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:09:36.445580 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e8be1-fa10-491d-85e8-406a631e9d2e None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:36.456095 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e8be1-fa10-491d-85e8-406a631e9d2e None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:36.461450 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aaf81c-ffb4-408f-9fcb-384f9f5cb837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:36.462129 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aaf81c-ffb4-408f-9fcb-384f9f5cb837 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:36.462413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aaf81c-ffb4-408f-9fcb-384f9f5cb837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:36.462699 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aaf81c-ffb4-408f-9fcb-384f9f5cb837 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:36.466084 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aaf81c-ffb4-408f-9fcb-384f9f5cb837 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:09:36.466084 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aaf81c-ffb4-408f-9fcb-384f9f5cb837 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:36.466084 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aaf81c-ffb4-408f-9fcb-384f9f5cb837 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:36.477001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1370/2765] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:09:36.481486 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec680421-ee39-4105-a8d3-746eface129c None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:36.490913 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec680421-ee39-4105-a8d3-746eface129c None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:36.496679 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3654f6d-15c5-42e3-bdbe-68fd136f09f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:36.497300 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3654f6d-15c5-42e3-bdbe-68fd136f09f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:36.497596 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3654f6d-15c5-42e3-bdbe-68fd136f09f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:36.497868 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3654f6d-15c5-42e3-bdbe-68fd136f09f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:36.498506 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3654f6d-15c5-42e3-bdbe-68fd136f09f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:36.500462 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3654f6d-15c5-42e3-bdbe-68fd136f09f8 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:36.512759 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:09:36.589136 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1396/2766] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:09:36 2022] POST /identity/v3/projects => generated 399 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:09:36.593298 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aaf81c-ffb4-408f-9fcb-384f9f5cb837 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:36.603767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aaf81c-ffb4-408f-9fcb-384f9f5cb837 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:36.669798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9cc9d-7234-4ed2-971c-e640e4c5e953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:36.670413 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9cc9d-7234-4ed2-971c-e640e4c5e953 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:36.670749 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9cc9d-7234-4ed2-971c-e640e4c5e953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:36.671138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9cc9d-7234-4ed2-971c-e640e4c5e953 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:36.672155 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9cc9d-7234-4ed2-971c-e640e4c5e953 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:36.675354 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9cc9d-7234-4ed2-971c-e640e4c5e953 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:36.782272 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1371/2767] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:09:36 2022] POST /identity/v3/users => generated 300 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:09:36.786415 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3654f6d-15c5-42e3-bdbe-68fd136f09f8 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:36.797854 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3654f6d-15c5-42e3-bdbe-68fd136f09f8 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:36.802450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e353a9e-1c21-4c44-bee4-3f4acf1a05a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:36.803073 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e353a9e-1c21-4c44-bee4-3f4acf1a05a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:36.803344 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e353a9e-1c21-4c44-bee4-3f4acf1a05a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:36.803609 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e353a9e-1c21-4c44-bee4-3f4acf1a05a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:36.804237 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e353a9e-1c21-4c44-bee4-3f4acf1a05a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:36.806447 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e353a9e-1c21-4c44-bee4-3f4acf1a05a1 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:36.816071 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1397/2768] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:09:36.820238 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9cc9d-7234-4ed2-971c-e640e4c5e953 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:36.832888 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9cc9d-7234-4ed2-971c-e640e4c5e953 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:36.838099 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518013fe-3b77-48f9-98cc-95d357049ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:36.838792 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518013fe-3b77-48f9-98cc-95d357049ecf admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:36.839116 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518013fe-3b77-48f9-98cc-95d357049ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:36.839445 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518013fe-3b77-48f9-98cc-95d357049ecf admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:36.840055 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518013fe-3b77-48f9-98cc-95d357049ecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:36.841688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518013fe-3b77-48f9-98cc-95d357049ecf admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:36.853343 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1372/2769] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:09:36.857723 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e353a9e-1c21-4c44-bee4-3f4acf1a05a1 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:36.870182 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e353a9e-1c21-4c44-bee4-3f4acf1a05a1 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:36.875959 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91841e-afc5-4b97-b9a0-2b93cef51382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:36.876620 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91841e-afc5-4b97-b9a0-2b93cef51382 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:36.876893 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91841e-afc5-4b97-b9a0-2b93cef51382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:36.877163 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91841e-afc5-4b97-b9a0-2b93cef51382 admin admin] PATH_INFO: `/v3/projects/e0294a7ed1d244a3b7195e45e0daa4f0/users/2e9621118d044cfca99095f95a22c2ef/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:36.892333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa91841e-afc5-4b97-b9a0-2b93cef51382 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0294a7ed1d244a3b7195e45e0daa4f0, user_id=2e9621118d044cfca99095f95a22c2ef, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:36.895063 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa91841e-afc5-4b97-b9a0-2b93cef51382 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:36.965002 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1398/2770] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:09:36 2022] PUT /identity/v3/projects/e0294a7ed1d244a3b7195e45e0daa4f0/users/2e9621118d044cfca99095f95a22c2ef/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:09:36.978988 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cfa61-b3f2-4175-b350-cf36079be00d None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:36.979369 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cfa61-b3f2-4175-b350-cf36079be00d None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:36.979714 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cfa61-b3f2-4175-b350-cf36079be00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:37.015016 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-499cfa61-b3f2-4175-b350-cf36079be00d None None] MFA Rules not processed for user `2e9621118d044cfca99095f95a22c2ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:09:37.016249 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-499cfa61-b3f2-4175-b350-cf36079be00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:37.086103 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1373/2771] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:09:36 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:09:37.097635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91841e-afc5-4b97-b9a0-2b93cef51382 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:37.099415 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-aa91841e-afc5-4b97-b9a0-2b93cef51382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:37.157563 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91841e-afc5-4b97-b9a0-2b93cef51382 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:37.161224 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7d97b-2036-4e8e-b9f6-c37446ec521c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:37.161835 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7d97b-2036-4e8e-b9f6-c37446ec521c service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:37.162146 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7d97b-2036-4e8e-b9f6-c37446ec521c service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:37.162386 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7d97b-2036-4e8e-b9f6-c37446ec521c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:37.172402 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7d97b-2036-4e8e-b9f6-c37446ec521c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:37.173870 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7d97b-2036-4e8e-b9f6-c37446ec521c service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:37.183059 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1399/2772] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:09:37 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:09:46.143850 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499cfa61-b3f2-4175-b350-cf36079be00d None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:46.188508 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:09:46.208870 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499cfa61-b3f2-4175-b350-cf36079be00d None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:46.213323 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8b5c0-eed4-4f4a-8c20-fcc5d6cef0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:46.213943 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8b5c0-eed4-4f4a-8c20-fcc5d6cef0f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:46.214216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8b5c0-eed4-4f4a-8c20-fcc5d6cef0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:46.214483 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8b5c0-eed4-4f4a-8c20-fcc5d6cef0f4 admin admin] PATH_INFO: `/v3/users/2e9621118d044cfca99095f95a22c2ef` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:46.215628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f8b5c0-eed4-4f4a-8c20-fcc5d6cef0f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e9621118d044cfca99095f95a22c2ef)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:46.218178 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f8b5c0-eed4-4f4a-8c20-fcc5d6cef0f4 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:46.384358 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d2f8b5c0-eed4-4f4a-8c20-fcc5d6cef0f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e9621118d044cfca99095f95a22c2ef'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:46.392152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d2f8b5c0-eed4-4f4a-8c20-fcc5d6cef0f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e9621118d044cfca99095f95a22c2ef'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:46.403997 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d2f8b5c0-eed4-4f4a-8c20-fcc5d6cef0f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e9621118d044cfca99095f95a22c2ef'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:46.477455 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d2f8b5c0-eed4-4f4a-8c20-fcc5d6cef0f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e9621118d044cfca99095f95a22c2ef'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:46.478088 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-d2f8b5c0-eed4-4f4a-8c20-fcc5d6cef0f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e9621118d044cfca99095f95a22c2ef'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:46.484697 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1374/2773] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:09:46 2022] DELETE /identity/v3/users/2e9621118d044cfca99095f95a22c2ef => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:09:46.493528 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c43f13-1653-4447-8515-778288b58f1d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:46.494072 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c43f13-1653-4447-8515-778288b58f1d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:46.494072 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c43f13-1653-4447-8515-778288b58f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:46.516734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-55c43f13-1653-4447-8515-778288b58f1d None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:09:46.520053 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-55c43f13-1653-4447-8515-778288b58f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:46.581521 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1400/2774] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:09:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:09:46.592019 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8b5c0-eed4-4f4a-8c20-fcc5d6cef0f4 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:46.593578 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d2f8b5c0-eed4-4f4a-8c20-fcc5d6cef0f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:46.649547 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8b5c0-eed4-4f4a-8c20-fcc5d6cef0f4 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:46.653066 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97efd2-098b-4fbc-b8e0-7132b908f220 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:46.653665 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97efd2-098b-4fbc-b8e0-7132b908f220 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:46.654550 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97efd2-098b-4fbc-b8e0-7132b908f220 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:46.655175 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97efd2-098b-4fbc-b8e0-7132b908f220 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:46.663856 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed97efd2-098b-4fbc-b8e0-7132b908f220 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:46.665273 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed97efd2-098b-4fbc-b8e0-7132b908f220 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:46.675463 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1375/2775] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:09:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:09:47.035002 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c43f13-1653-4447-8515-778288b58f1d None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:47.036512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-55c43f13-1653-4447-8515-778288b58f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:47.050962 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c43f13-1653-4447-8515-778288b58f1d None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:47.056003 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23761a7-7f11-4335-b9f2-9cf7acb2cb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:47.056631 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23761a7-7f11-4335-b9f2-9cf7acb2cb1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:47.056904 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23761a7-7f11-4335-b9f2-9cf7acb2cb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:47.057173 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23761a7-7f11-4335-b9f2-9cf7acb2cb1b admin admin] PATH_INFO: `/v3/projects/e0294a7ed1d244a3b7195e45e0daa4f0` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:47.058283 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23761a7-7f11-4335-b9f2-9cf7acb2cb1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0294a7ed1d244a3b7195e45e0daa4f0)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:47.060820 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23761a7-7f11-4335-b9f2-9cf7acb2cb1b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:47.164092 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c23761a7-7f11-4335-b9f2-9cf7acb2cb1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0294a7ed1d244a3b7195e45e0daa4f0', 'request_id': 'req-c23761a7-7f11-4335-b9f2-9cf7acb2cb1b'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:47.167568 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:09:47.232420 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c23761a7-7f11-4335-b9f2-9cf7acb2cb1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0294a7ed1d244a3b7195e45e0daa4f0', 'request_id': 'req-c23761a7-7f11-4335-b9f2-9cf7acb2cb1b'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:47.238602 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c23761a7-7f11-4335-b9f2-9cf7acb2cb1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0294a7ed1d244a3b7195e45e0daa4f0', 'request_id': 'req-c23761a7-7f11-4335-b9f2-9cf7acb2cb1b'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:47.250538 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c23761a7-7f11-4335-b9f2-9cf7acb2cb1b admin admin] The token cache is being invalidate because project e0294a7ed1d244a3b7195e45e0daa4f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:09:47.250860 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c23761a7-7f11-4335-b9f2-9cf7acb2cb1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:47.251514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-c23761a7-7f11-4335-b9f2-9cf7acb2cb1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:09:47.252994 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1401/2776] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:09:47 2022] DELETE /identity/v3/projects/e0294a7ed1d244a3b7195e45e0daa4f0 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:09:48.580265 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bf5e2-5d58-4640-b304-205d215f5daf None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:48.580763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bf5e2-5d58-4640-b304-205d215f5daf None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:48.580859 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bf5e2-5d58-4640-b304-205d215f5daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:48.600100 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-e51bf5e2-5d58-4640-b304-205d215f5daf None None] MFA Rules not processed for user `aa4f534ee1ab4e639e99b9e4ebbe8363`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:09:48.601304 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-e51bf5e2-5d58-4640-b304-205d215f5daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:48.655027 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1376/2777] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:09:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:09:51.039616 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23761a7-7f11-4335-b9f2-9cf7acb2cb1b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:09:51.040997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c23761a7-7f11-4335-b9f2-9cf7acb2cb1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:51.111343 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23761a7-7f11-4335-b9f2-9cf7acb2cb1b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:09:51.116432 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e85cd48-db78-456f-a354-44e76289a0f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cb1d85a338e40f18e61887a1becb906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:09:51.122989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85cd48-db78-456f-a354-44e76289a0f5 service placement] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:51.122989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85cd48-db78-456f-a354-44e76289a0f5 service placement] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:51.122989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85cd48-db78-456f-a354-44e76289a0f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:51.122989 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5e85cd48-db78-456f-a354-44e76289a0f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:51.130856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e85cd48-db78-456f-a354-44e76289a0f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:09:51.132415 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e85cd48-db78-456f-a354-44e76289a0f5 service placement] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:09:51.143044 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1402/2778] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:09:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:09:56.509539 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566d5bf-21b3-4adb-9ce8-8f3e1708f132 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:09:56.510037 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566d5bf-21b3-4adb-9ce8-8f3e1708f132 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:09:56.510138 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566d5bf-21b3-4adb-9ce8-8f3e1708f132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:09:56.525704 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:09:56.531115 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-a566d5bf-21b3-4adb-9ce8-8f3e1708f132 None None] MFA Rules not processed for user `6ae921c806734b0a8218ea7453e8bd4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:09:56.532304 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a566d5bf-21b3-4adb-9ce8-8f3e1708f132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:09:56.586640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1377/2779] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:09:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:10:01.138263 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=38,keystone:DELETE=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:10:06.614146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=22 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:10:08.775067 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e85cd48-db78-456f-a354-44e76289a0f5 service placement] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:10:08.776495 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5e85cd48-db78-456f-a354-44e76289a0f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:10:08.834337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e85cd48-db78-456f-a354-44e76289a0f5 service placement] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:10:08.837976 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5e32b-5558-4a59-8ccd-e285943c109f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:10:08.838579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5e32b-5558-4a59-8ccd-e285943c109f service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:10:08.838953 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5e32b-5558-4a59-8ccd-e285943c109f service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:10:08.839230 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5e32b-5558-4a59-8ccd-e285943c109f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:10:08.840963 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-79c5e32b-5558-4a59-8ccd-e285943c109f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:10:08.903932 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5e32b-5558-4a59-8ccd-e285943c109f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:10:08.905350 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5e32b-5558-4a59-8ccd-e285943c109f service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:10:08.915763 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1403/2780] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:10:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:10:18.911183 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=50 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:10:58.453231 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566d5bf-21b3-4adb-9ce8-8f3e1708f132 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:10:58.455009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-a566d5bf-21b3-4adb-9ce8-8f3e1708f132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:10:58.468075 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566d5bf-21b3-4adb-9ce8-8f3e1708f132 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:10:58.471630 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127bfbc3-5e09-4ef5-bcfc-bfab3dffc09d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:10:58.472230 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127bfbc3-5e09-4ef5-bcfc-bfab3dffc09d service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:10:58.472504 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127bfbc3-5e09-4ef5-bcfc-bfab3dffc09d service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:10:58.472774 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127bfbc3-5e09-4ef5-bcfc-bfab3dffc09d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:10:58.475508 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-127bfbc3-5e09-4ef5-bcfc-bfab3dffc09d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:10:58.525554 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5e32b-5558-4a59-8ccd-e285943c109f service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:10:58.534218 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5e32b-5558-4a59-8ccd-e285943c109f service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:10:58.539348 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c03b0f-d875-4902-8882-8d19eee3192d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:10:58.540109 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c03b0f-d875-4902-8882-8d19eee3192d service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:10:58.540298 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c03b0f-d875-4902-8882-8d19eee3192d service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:10:58.540571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c03b0f-d875-4902-8882-8d19eee3192d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:10:58.542342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-94c03b0f-d875-4902-8882-8d19eee3192d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:10:58.552060 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127bfbc3-5e09-4ef5-bcfc-bfab3dffc09d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:10:58.554637 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127bfbc3-5e09-4ef5-bcfc-bfab3dffc09d service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:10:58.556602 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c03b0f-d875-4902-8882-8d19eee3192d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:10:58.558167 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c03b0f-d875-4902-8882-8d19eee3192d service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:10:58.566500 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1378/2781] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:10:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:10:58.570020 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1404/2782] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:10:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:11:06.588408 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1d9f2-d51d-4c85-9a5e-985a39cfce27 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:11:06.588892 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1d9f2-d51d-4c85-9a5e-985a39cfce27 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:11:06.592043 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1d9f2-d51d-4c85-9a5e-985a39cfce27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:11:06.608505 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-5ad1d9f2-d51d-4c85-9a5e-985a39cfce27 None None] MFA Rules not processed for user `6ae921c806734b0a8218ea7453e8bd4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:11:06.609890 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5ad1d9f2-d51d-4c85-9a5e-985a39cfce27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:11:06.617072 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1379/2783] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:11:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:11:08.564217 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=6 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:11:16.602956 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=37 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:12:12.768172 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c03b0f-d875-4902-8882-8d19eee3192d service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:12:12.769791 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-94c03b0f-d875-4902-8882-8d19eee3192d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:12:12.785796 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c03b0f-d875-4902-8882-8d19eee3192d service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:12:12.792476 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e00c26-ddba-45af-b018-2fe8fbb453e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:12:12.793165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e00c26-ddba-45af-b018-2fe8fbb453e7 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:12:12.793520 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e00c26-ddba-45af-b018-2fe8fbb453e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:12:12.793857 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e00c26-ddba-45af-b018-2fe8fbb453e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:12:12.795708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-70e00c26-ddba-45af-b018-2fe8fbb453e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:12:12.810467 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e00c26-ddba-45af-b018-2fe8fbb453e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:12:12.812947 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e00c26-ddba-45af-b018-2fe8fbb453e7 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:12:12.826635 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1405/2784] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:12:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:12:15.125525 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61558d77-ebd8-4a86-882f-a3fcdc3eaf0f None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:12:15.126023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61558d77-ebd8-4a86-882f-a3fcdc3eaf0f None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:12:15.126023 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61558d77-ebd8-4a86-882f-a3fcdc3eaf0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:12:15.145776 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-61558d77-ebd8-4a86-882f-a3fcdc3eaf0f None None] MFA Rules not processed for user `aa4f534ee1ab4e639e99b9e4ebbe8363`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:12:15.147010 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-61558d77-ebd8-4a86-882f-a3fcdc3eaf0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:12:15.154779 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1380/2785] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:12:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:12:16.258982 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e00c26-ddba-45af-b018-2fe8fbb453e7 service nova] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:12:16.272131 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e00c26-ddba-45af-b018-2fe8fbb453e7 service nova] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:12:16.278193 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa968f-88ea-4253-9527-e3ab06ba9a40 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cb1d85a338e40f18e61887a1becb906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:12:16.278880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa968f-88ea-4253-9527-e3ab06ba9a40 service placement] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:12:16.279159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa968f-88ea-4253-9527-e3ab06ba9a40 service placement] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:12:16.279429 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa968f-88ea-4253-9527-e3ab06ba9a40 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:12:16.281514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-abfa968f-88ea-4253-9527-e3ab06ba9a40 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:12:16.301100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa968f-88ea-4253-9527-e3ab06ba9a40 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:12:16.302956 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa968f-88ea-4253-9527-e3ab06ba9a40 service placement] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:12:16.319465 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1406/2786] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:12:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:12:19.757129 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61558d77-ebd8-4a86-882f-a3fcdc3eaf0f None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:12:19.768004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61558d77-ebd8-4a86-882f-a3fcdc3eaf0f None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:12:19.772151 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34576e59-39f4-4187-974d-56e1a3697a76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:12:19.773019 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34576e59-39f4-4187-974d-56e1a3697a76 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:12:19.773879 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34576e59-39f4-4187-974d-56e1a3697a76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:12:19.774948 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34576e59-39f4-4187-974d-56e1a3697a76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:12:19.777752 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-34576e59-39f4-4187-974d-56e1a3697a76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:12:19.793726 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34576e59-39f4-4187-974d-56e1a3697a76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:12:19.793726 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34576e59-39f4-4187-974d-56e1a3697a76 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:12:19.799802 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1381/2787] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:12:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:12:23.057357 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701be7fb-0a35-4023-b962-7dd7b64b2fd1 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:12:23.057850 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701be7fb-0a35-4023-b962-7dd7b64b2fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:12:23.058078 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701be7fb-0a35-4023-b962-7dd7b64b2fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:12:23.078754 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-701be7fb-0a35-4023-b962-7dd7b64b2fd1 None None] MFA Rules not processed for user `6ae921c806734b0a8218ea7453e8bd4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:12:23.079974 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-701be7fb-0a35-4023-b962-7dd7b64b2fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:12:23.087949 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1407/2788] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:12:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:12:23.409910 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34576e59-39f4-4187-974d-56e1a3697a76 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:12:23.419232 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34576e59-39f4-4187-974d-56e1a3697a76 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:12:23.423004 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abec312-e206-4c3f-8f0e-b9fb92f32d63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:12:23.423605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abec312-e206-4c3f-8f0e-b9fb92f32d63 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:12:23.423884 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abec312-e206-4c3f-8f0e-b9fb92f32d63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:12:23.424158 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abec312-e206-4c3f-8f0e-b9fb92f32d63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:12:23.426037 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-0abec312-e206-4c3f-8f0e-b9fb92f32d63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:12:23.436782 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abec312-e206-4c3f-8f0e-b9fb92f32d63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:12:23.438277 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abec312-e206-4c3f-8f0e-b9fb92f32d63 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:12:23.446671 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1382/2789] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:12:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:12:32.688036 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701be7fb-0a35-4023-b962-7dd7b64b2fd1 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:12:32.700523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701be7fb-0a35-4023-b962-7dd7b64b2fd1 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:12:32.706175 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6480b087-97f5-4fc8-bd36-bd7d0f9372e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:12:32.706780 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480b087-97f5-4fc8-bd36-bd7d0f9372e3 service nova] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:12:32.707059 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480b087-97f5-4fc8-bd36-bd7d0f9372e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:12:32.707329 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480b087-97f5-4fc8-bd36-bd7d0f9372e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:12:32.709053 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6480b087-97f5-4fc8-bd36-bd7d0f9372e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:12:32.721603 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6480b087-97f5-4fc8-bd36-bd7d0f9372e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:12:32.723197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6480b087-97f5-4fc8-bd36-bd7d0f9372e3 service nova] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:12:32.734242 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1408/2790] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:12:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:12:33.442853 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=15 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:12:38.189975 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abec312-e206-4c3f-8f0e-b9fb92f32d63 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:12:38.203323 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abec312-e206-4c3f-8f0e-b9fb92f32d63 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:12:38.209338 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64bc33-b23c-43ca-b127-d176e821073b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:12:38.210139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64bc33-b23c-43ca-b127-d176e821073b service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:12:38.210955 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64bc33-b23c-43ca-b127-d176e821073b service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:12:38.211394 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64bc33-b23c-43ca-b127-d176e821073b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:12:38.213361 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5d64bc33-b23c-43ca-b127-d176e821073b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:12:38.226846 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d64bc33-b23c-43ca-b127-d176e821073b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:12:38.228763 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d64bc33-b23c-43ca-b127-d176e821073b service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:12:38.241740 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1383/2791] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:12:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:12:42.730120 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=21 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:12:46.463933 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6480b087-97f5-4fc8-bd36-bd7d0f9372e3 service nova] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:12:46.473282 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6480b087-97f5-4fc8-bd36-bd7d0f9372e3 service nova] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:12:46.477524 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd486cf-7578-4a61-9a49-f30d4857f70d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:12:46.478351 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd486cf-7578-4a61-9a49-f30d4857f70d service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:12:46.478801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd486cf-7578-4a61-9a49-f30d4857f70d service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:12:46.479212 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd486cf-7578-4a61-9a49-f30d4857f70d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:12:46.481382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ebd486cf-7578-4a61-9a49-f30d4857f70d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:12:46.548150 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd486cf-7578-4a61-9a49-f30d4857f70d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:12:46.549573 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd486cf-7578-4a61-9a49-f30d4857f70d service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:12:46.559628 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1409/2792] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:12:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:12:48.236159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=6 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:12:52.275940 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64bc33-b23c-43ca-b127-d176e821073b service nova] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:12:52.277659 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5d64bc33-b23c-43ca-b127-d176e821073b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:12:52.292723 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64bc33-b23c-43ca-b127-d176e821073b service nova] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:12:52.297218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda2e9e-1a79-4125-8ced-5ab8c2c58d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:12:52.297844 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda2e9e-1a79-4125-8ced-5ab8c2c58d84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:12:52.298119 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda2e9e-1a79-4125-8ced-5ab8c2c58d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:12:52.298393 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda2e9e-1a79-4125-8ced-5ab8c2c58d84 admin admin] PATH_INFO: `/v3/users/32b5ea2215fd4c38aabaf375db991d06` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:12:52.299539 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda2e9e-1a79-4125-8ced-5ab8c2c58d84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32b5ea2215fd4c38aabaf375db991d06)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:12:52.301461 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda2e9e-1a79-4125-8ced-5ab8c2c58d84 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:12:52.515282 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8cda2e9e-1a79-4125-8ced-5ab8c2c58d84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32b5ea2215fd4c38aabaf375db991d06'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:12:52.522128 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8cda2e9e-1a79-4125-8ced-5ab8c2c58d84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32b5ea2215fd4c38aabaf375db991d06'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:12:52.534786 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8cda2e9e-1a79-4125-8ced-5ab8c2c58d84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32b5ea2215fd4c38aabaf375db991d06'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:12:52.649922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8cda2e9e-1a79-4125-8ced-5ab8c2c58d84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32b5ea2215fd4c38aabaf375db991d06'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:12:52.650470 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-8cda2e9e-1a79-4125-8ced-5ab8c2c58d84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32b5ea2215fd4c38aabaf375db991d06'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:12:52.656986 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1384/2793] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:12:52 2022] DELETE /identity/v3/users/32b5ea2215fd4c38aabaf375db991d06 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:12:52.939619 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd486cf-7578-4a61-9a49-f30d4857f70d service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:12:52.953442 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd486cf-7578-4a61-9a49-f30d4857f70d service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:12:53.015968 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04367cbf-e72c-455c-8b20-99dc88380d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:12:53.016641 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04367cbf-e72c-455c-8b20-99dc88380d27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:12:53.016923 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04367cbf-e72c-455c-8b20-99dc88380d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:12:53.017195 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04367cbf-e72c-455c-8b20-99dc88380d27 admin admin] PATH_INFO: `/v3/projects/49b0d0136b404c68902274c0e571fb3a` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:12:53.018311 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04367cbf-e72c-455c-8b20-99dc88380d27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49b0d0136b404c68902274c0e571fb3a)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:12:53.020243 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04367cbf-e72c-455c-8b20-99dc88380d27 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:12:53.195010 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-04367cbf-e72c-455c-8b20-99dc88380d27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49b0d0136b404c68902274c0e571fb3a', 'request_id': 'req-04367cbf-e72c-455c-8b20-99dc88380d27'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:12:53.297095 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-04367cbf-e72c-455c-8b20-99dc88380d27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49b0d0136b404c68902274c0e571fb3a', 'request_id': 'req-04367cbf-e72c-455c-8b20-99dc88380d27'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:12:53.303443 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-04367cbf-e72c-455c-8b20-99dc88380d27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49b0d0136b404c68902274c0e571fb3a', 'request_id': 'req-04367cbf-e72c-455c-8b20-99dc88380d27'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:12:53.315420 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-04367cbf-e72c-455c-8b20-99dc88380d27 admin admin] The token cache is being invalidate because project 49b0d0136b404c68902274c0e571fb3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:12:53.320360 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-04367cbf-e72c-455c-8b20-99dc88380d27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:12:53.320360 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-04367cbf-e72c-455c-8b20-99dc88380d27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:12:53.320360 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1410/2794] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:12:52 2022] DELETE /identity/v3/projects/49b0d0136b404c68902274c0e571fb3a => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:12:53.321956 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda2e9e-1a79-4125-8ced-5ab8c2c58d84 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:12:53.323412 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-8cda2e9e-1a79-4125-8ced-5ab8c2c58d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:12:53.336847 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda2e9e-1a79-4125-8ced-5ab8c2c58d84 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:12:53.393159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbed287-8951-4c3b-bc44-2547c9769921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:12:53.393779 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbed287-8951-4c3b-bc44-2547c9769921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:12:53.394053 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbed287-8951-4c3b-bc44-2547c9769921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:12:53.394318 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbed287-8951-4c3b-bc44-2547c9769921 admin admin] PATH_INFO: `/v3/users/3c3b85078428455da0c5be75378b5cd7` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:12:53.395455 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbed287-8951-4c3b-bc44-2547c9769921 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c3b85078428455da0c5be75378b5cd7)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:12:53.397371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbed287-8951-4c3b-bc44-2547c9769921 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:12:53.657903 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5cbed287-8951-4c3b-bc44-2547c9769921 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c3b85078428455da0c5be75378b5cd7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:12:53.665330 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5cbed287-8951-4c3b-bc44-2547c9769921 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c3b85078428455da0c5be75378b5cd7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:12:53.676919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5cbed287-8951-4c3b-bc44-2547c9769921 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c3b85078428455da0c5be75378b5cd7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:12:53.678565 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:12:53.817334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5cbed287-8951-4c3b-bc44-2547c9769921 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c3b85078428455da0c5be75378b5cd7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:12:53.818147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-5cbed287-8951-4c3b-bc44-2547c9769921 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c3b85078428455da0c5be75378b5cd7'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:12:53.824853 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1385/2795] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:12:53 2022] DELETE /identity/v3/users/3c3b85078428455da0c5be75378b5cd7 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:12:54.113468 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04367cbf-e72c-455c-8b20-99dc88380d27 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:12:54.114996 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-04367cbf-e72c-455c-8b20-99dc88380d27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:12:54.128241 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04367cbf-e72c-455c-8b20-99dc88380d27 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:12:54.149484 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:12:54.184731 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4cb29-2a06-4fc8-a9a9-e14df329636d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:12:54.185287 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4cb29-2a06-4fc8-a9a9-e14df329636d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:12:54.185564 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4cb29-2a06-4fc8-a9a9-e14df329636d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:12:54.185833 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4cb29-2a06-4fc8-a9a9-e14df329636d admin admin] PATH_INFO: `/v3/projects/86899cff62834c7fb481b7c87ca10dc6` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:12:54.186967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d4cb29-2a06-4fc8-a9a9-e14df329636d admin admin] RBAC: Authorizing `identity:delete_project(project_id=86899cff62834c7fb481b7c87ca10dc6)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:12:54.188855 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d4cb29-2a06-4fc8-a9a9-e14df329636d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:12:54.355628 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-31d4cb29-2a06-4fc8-a9a9-e14df329636d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86899cff62834c7fb481b7c87ca10dc6', 'request_id': 'req-31d4cb29-2a06-4fc8-a9a9-e14df329636d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:12:54.445993 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-31d4cb29-2a06-4fc8-a9a9-e14df329636d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86899cff62834c7fb481b7c87ca10dc6', 'request_id': 'req-31d4cb29-2a06-4fc8-a9a9-e14df329636d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:12:54.452142 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-31d4cb29-2a06-4fc8-a9a9-e14df329636d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86899cff62834c7fb481b7c87ca10dc6', 'request_id': 'req-31d4cb29-2a06-4fc8-a9a9-e14df329636d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:12:54.463446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-31d4cb29-2a06-4fc8-a9a9-e14df329636d admin admin] The token cache is being invalidate because project 86899cff62834c7fb481b7c87ca10dc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:12:54.463731 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-31d4cb29-2a06-4fc8-a9a9-e14df329636d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:12:54.465670 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-31d4cb29-2a06-4fc8-a9a9-e14df329636d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:12:54.465903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1411/2796] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:12:54 2022] DELETE /identity/v3/projects/86899cff62834c7fb481b7c87ca10dc6 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:13:03.911841 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:13:04.460450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:13:40.413681 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbed287-8951-4c3b-bc44-2547c9769921 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:40.415168 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5cbed287-8951-4c3b-bc44-2547c9769921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:40.478625 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbed287-8951-4c3b-bc44-2547c9769921 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:40.525776 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f707ea-c405-41ac-9ac4-6ecb0c590d5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:40.526373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f707ea-c405-41ac-9ac4-6ecb0c590d5e service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:40.526653 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f707ea-c405-41ac-9ac4-6ecb0c590d5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:40.526952 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f707ea-c405-41ac-9ac4-6ecb0c590d5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:40.528759 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-39f707ea-c405-41ac-9ac4-6ecb0c590d5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:40.595217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f707ea-c405-41ac-9ac4-6ecb0c590d5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:40.596660 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f707ea-c405-41ac-9ac4-6ecb0c590d5e service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:40.606955 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1386/2797] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:13:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:13:41.506795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4cb29-2a06-4fc8-a9a9-e14df329636d admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:41.511272 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-31d4cb29-2a06-4fc8-a9a9-e14df329636d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:41.585977 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4cb29-2a06-4fc8-a9a9-e14df329636d admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:41.590943 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de207f2d-1f1e-4496-8d12-00718fd76f07 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cb1d85a338e40f18e61887a1becb906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:41.591540 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de207f2d-1f1e-4496-8d12-00718fd76f07 service placement] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:41.591813 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de207f2d-1f1e-4496-8d12-00718fd76f07 service placement] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:41.592082 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de207f2d-1f1e-4496-8d12-00718fd76f07 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:41.593809 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-de207f2d-1f1e-4496-8d12-00718fd76f07 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:41.605696 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de207f2d-1f1e-4496-8d12-00718fd76f07 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:41.607283 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de207f2d-1f1e-4496-8d12-00718fd76f07 service placement] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:41.617097 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1412/2798] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:13:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:13:41.647572 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f707ea-c405-41ac-9ac4-6ecb0c590d5e service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:41.649497 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-39f707ea-c405-41ac-9ac4-6ecb0c590d5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:41.662374 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f707ea-c405-41ac-9ac4-6ecb0c590d5e service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:41.667223 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af748d1-f5cd-4e75-91ac-76800b2badf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:41.667985 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af748d1-f5cd-4e75-91ac-76800b2badf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:41.668327 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af748d1-f5cd-4e75-91ac-76800b2badf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:41.668590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af748d1-f5cd-4e75-91ac-76800b2badf7 admin admin] PATH_INFO: `/v3/users/6ae921c806734b0a8218ea7453e8bd4e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:41.669696 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af748d1-f5cd-4e75-91ac-76800b2badf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ae921c806734b0a8218ea7453e8bd4e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:41.671645 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af748d1-f5cd-4e75-91ac-76800b2badf7 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:41.908241 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9af748d1-f5cd-4e75-91ac-76800b2badf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ae921c806734b0a8218ea7453e8bd4e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:41.916009 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9af748d1-f5cd-4e75-91ac-76800b2badf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ae921c806734b0a8218ea7453e8bd4e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:41.920361 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:13:41.928710 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9af748d1-f5cd-4e75-91ac-76800b2badf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ae921c806734b0a8218ea7453e8bd4e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:42.044530 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9af748d1-f5cd-4e75-91ac-76800b2badf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ae921c806734b0a8218ea7453e8bd4e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:42.045151 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-9af748d1-f5cd-4e75-91ac-76800b2badf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ae921c806734b0a8218ea7453e8bd4e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:42.052011 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1387/2799] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:13:41 2022] DELETE /identity/v3/users/6ae921c806734b0a8218ea7453e8bd4e => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:13:42.292771 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de207f2d-1f1e-4496-8d12-00718fd76f07 service placement] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:42.304758 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de207f2d-1f1e-4496-8d12-00718fd76f07 service placement] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:42.366157 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3ff30-0d6f-4ef8-aa3e-1220ba53d743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:42.366842 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3ff30-0d6f-4ef8-aa3e-1220ba53d743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:42.367121 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3ff30-0d6f-4ef8-aa3e-1220ba53d743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:42.367391 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3ff30-0d6f-4ef8-aa3e-1220ba53d743 admin admin] PATH_INFO: `/v3/projects/261177a9929f4df4bff1d0c009b1b3e2` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:42.368496 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b3ff30-0d6f-4ef8-aa3e-1220ba53d743 admin admin] RBAC: Authorizing `identity:delete_project(project_id=261177a9929f4df4bff1d0c009b1b3e2)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:42.370460 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b3ff30-0d6f-4ef8-aa3e-1220ba53d743 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:42.485643 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e8b3ff30-0d6f-4ef8-aa3e-1220ba53d743 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '261177a9929f4df4bff1d0c009b1b3e2', 'request_id': 'req-e8b3ff30-0d6f-4ef8-aa3e-1220ba53d743'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:42.536575 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e8b3ff30-0d6f-4ef8-aa3e-1220ba53d743 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '261177a9929f4df4bff1d0c009b1b3e2', 'request_id': 'req-e8b3ff30-0d6f-4ef8-aa3e-1220ba53d743'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:42.542687 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e8b3ff30-0d6f-4ef8-aa3e-1220ba53d743 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '261177a9929f4df4bff1d0c009b1b3e2', 'request_id': 'req-e8b3ff30-0d6f-4ef8-aa3e-1220ba53d743'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:42.554474 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e8b3ff30-0d6f-4ef8-aa3e-1220ba53d743 admin admin] The token cache is being invalidate because project 261177a9929f4df4bff1d0c009b1b3e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:13:42.554785 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e8b3ff30-0d6f-4ef8-aa3e-1220ba53d743 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:42.555409 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-e8b3ff30-0d6f-4ef8-aa3e-1220ba53d743 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:42.556879 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1413/2800] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:13:42 2022] DELETE /identity/v3/projects/261177a9929f4df4bff1d0c009b1b3e2 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:13:42.561819 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d2988-5cd0-4cea-91b7-42330af94cb0 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:42.562170 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d2988-5cd0-4cea-91b7-42330af94cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:42.562452 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d2988-5cd0-4cea-91b7-42330af94cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:42.582478 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-703d2988-5cd0-4cea-91b7-42330af94cb0 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:13:42.583715 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-703d2988-5cd0-4cea-91b7-42330af94cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:42.641731 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1388/2801] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:13:42.650124 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad170664-3c3b-4329-b3f9-deea9b7d0b16 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:42.650408 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad170664-3c3b-4329-b3f9-deea9b7d0b16 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:42.650706 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad170664-3c3b-4329-b3f9-deea9b7d0b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:42.670708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-ad170664-3c3b-4329-b3f9-deea9b7d0b16 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:13:42.671903 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ad170664-3c3b-4329-b3f9-deea9b7d0b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:42.680325 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1414/2802] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:13:42.684391 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d2988-5cd0-4cea-91b7-42330af94cb0 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:42.693971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d2988-5cd0-4cea-91b7-42330af94cb0 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:42.698499 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cdf71e-1611-44cd-a439-4d60bce714d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:42.699115 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdf71e-1611-44cd-a439-4d60bce714d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:42.699418 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdf71e-1611-44cd-a439-4d60bce714d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:42.699691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdf71e-1611-44cd-a439-4d60bce714d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:42.700327 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cdf71e-1611-44cd-a439-4d60bce714d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:13:42.700627 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cdf71e-1611-44cd-a439-4d60bce714d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:42.702221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cdf71e-1611-44cd-a439-4d60bce714d3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:42.713887 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1389/2803] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:13:42.717992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad170664-3c3b-4329-b3f9-deea9b7d0b16 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:42.728630 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad170664-3c3b-4329-b3f9-deea9b7d0b16 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:42.733276 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f5fb4-89ef-4eb1-8a7b-80413a212347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:42.733863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f5fb4-89ef-4eb1-8a7b-80413a212347 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:42.734138 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f5fb4-89ef-4eb1-8a7b-80413a212347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:42.734406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f5fb4-89ef-4eb1-8a7b-80413a212347 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:42.735069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f5fb4-89ef-4eb1-8a7b-80413a212347 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:42.736987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f5fb4-89ef-4eb1-8a7b-80413a212347 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:42.748440 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:13:42.806557 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1415/2804] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:13:42 2022] POST /identity/v3/projects => generated 385 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:13:42.810780 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cdf71e-1611-44cd-a439-4d60bce714d3 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:42.820853 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cdf71e-1611-44cd-a439-4d60bce714d3 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:42.879316 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7ff1e-4c0a-4cb8-836e-3dc7514704f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:42.879911 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7ff1e-4c0a-4cb8-836e-3dc7514704f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:42.880189 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7ff1e-4c0a-4cb8-836e-3dc7514704f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:42.880461 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7ff1e-4c0a-4cb8-836e-3dc7514704f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:42.881099 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f7ff1e-4c0a-4cb8-836e-3dc7514704f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:42.883082 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f7ff1e-4c0a-4cb8-836e-3dc7514704f4 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:42.977113 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1390/2805] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:13:42 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:13:42.981129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f5fb4-89ef-4eb1-8a7b-80413a212347 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:42.992711 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f5fb4-89ef-4eb1-8a7b-80413a212347 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:42.998333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85b73d-8ff0-4745-ba1a-84a488900a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:42.998996 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85b73d-8ff0-4745-ba1a-84a488900a7e admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:43.000115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85b73d-8ff0-4745-ba1a-84a488900a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:43.000115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85b73d-8ff0-4745-ba1a-84a488900a7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:43.000333 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc85b73d-8ff0-4745-ba1a-84a488900a7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:43.001785 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc85b73d-8ff0-4745-ba1a-84a488900a7e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:43.012105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1416/2806] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:13:43.016218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7ff1e-4c0a-4cb8-836e-3dc7514704f4 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:43.027404 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7ff1e-4c0a-4cb8-836e-3dc7514704f4 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:43.032008 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3bc8f3-afde-416f-a198-b891f2c1f57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:43.032596 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bc8f3-afde-416f-a198-b891f2c1f57c admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:43.032900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bc8f3-afde-416f-a198-b891f2c1f57c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:43.033166 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bc8f3-afde-416f-a198-b891f2c1f57c admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:43.033794 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3bc8f3-afde-416f-a198-b891f2c1f57c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:43.036309 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3bc8f3-afde-416f-a198-b891f2c1f57c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:43.046780 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1391/2807] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:13:43.051065 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85b73d-8ff0-4745-ba1a-84a488900a7e admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:43.061125 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85b73d-8ff0-4745-ba1a-84a488900a7e admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:43.065719 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d2e33-245d-4b52-a179-0d956cc0037f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:43.066459 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d2e33-245d-4b52-a179-0d956cc0037f admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:43.066649 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d2e33-245d-4b52-a179-0d956cc0037f admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:43.066947 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d2e33-245d-4b52-a179-0d956cc0037f admin admin] PATH_INFO: `/v3/projects/0bcc6935200a4726acaa3f6d64d25043/users/07353ae77ab54bd9b4e7b6279ae9c071/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:43.081174 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2d2e33-245d-4b52-a179-0d956cc0037f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bcc6935200a4726acaa3f6d64d25043, user_id=07353ae77ab54bd9b4e7b6279ae9c071, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:43.084604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2d2e33-245d-4b52-a179-0d956cc0037f admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:43.183781 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1417/2808] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:13:43 2022] PUT /identity/v3/projects/0bcc6935200a4726acaa3f6d64d25043/users/07353ae77ab54bd9b4e7b6279ae9c071/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:13:43.195617 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8881fcf-d6c7-4b00-aca2-621760078361 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:43.196001 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8881fcf-d6c7-4b00-aca2-621760078361 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:43.196276 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8881fcf-d6c7-4b00-aca2-621760078361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:43.224662 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-c8881fcf-d6c7-4b00-aca2-621760078361 None None] MFA Rules not processed for user `07353ae77ab54bd9b4e7b6279ae9c071`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:13:43.225863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-c8881fcf-d6c7-4b00-aca2-621760078361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:43.258460 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:13:43.289969 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1392/2809] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:13:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:13:43.294581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d2e33-245d-4b52-a179-0d956cc0037f admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:43.358111 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d2e33-245d-4b52-a179-0d956cc0037f admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:43.362763 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d05c2d-8018-4ff7-b6d7-d2546d10010b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:43.363471 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d05c2d-8018-4ff7-b6d7-d2546d10010b admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:43.363628 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d05c2d-8018-4ff7-b6d7-d2546d10010b admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:43.363899 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d05c2d-8018-4ff7-b6d7-d2546d10010b admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:43.364522 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d05c2d-8018-4ff7-b6d7-d2546d10010b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:43.366408 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d05c2d-8018-4ff7-b6d7-d2546d10010b admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:43.531791 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1418/2810] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:13:43 2022] POST /identity/v3/projects => generated 387 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:13:43.535758 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8881fcf-d6c7-4b00-aca2-621760078361 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:43.546654 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8881fcf-d6c7-4b00-aca2-621760078361 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:43.608173 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff4c8d-79da-4868-b1d4-37694ea5afff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:43.608769 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff4c8d-79da-4868-b1d4-37694ea5afff admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:43.609042 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff4c8d-79da-4868-b1d4-37694ea5afff admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:43.609312 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff4c8d-79da-4868-b1d4-37694ea5afff admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:43.609942 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff4c8d-79da-4868-b1d4-37694ea5afff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:43.611907 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff4c8d-79da-4868-b1d4-37694ea5afff admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:43.701621 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1393/2811] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:13:43 2022] POST /identity/v3/users => generated 300 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:13:43.705548 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d05c2d-8018-4ff7-b6d7-d2546d10010b admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:43.715996 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d05c2d-8018-4ff7-b6d7-d2546d10010b admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:43.720600 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc3fbf-ce83-4157-826c-709247c42f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:43.721183 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc3fbf-ce83-4157-826c-709247c42f58 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:43.721452 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc3fbf-ce83-4157-826c-709247c42f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:43.721716 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc3fbf-ce83-4157-826c-709247c42f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:43.722359 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc3fbf-ce83-4157-826c-709247c42f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:43.724210 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc3fbf-ce83-4157-826c-709247c42f58 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:43.734471 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1419/2812] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:13:43.739035 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff4c8d-79da-4868-b1d4-37694ea5afff admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:43.749057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff4c8d-79da-4868-b1d4-37694ea5afff admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:43.754524 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb88099-6e9f-4b32-af14-8907cf6d24bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:43.755193 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb88099-6e9f-4b32-af14-8907cf6d24bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:43.755468 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb88099-6e9f-4b32-af14-8907cf6d24bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:43.755847 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb88099-6e9f-4b32-af14-8907cf6d24bd admin admin] PATH_INFO: `/v3/projects/0467340db5bb49ed83e8788b24ef2356/users/231a7e71361648a3a90c63534986b618/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:43.770745 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb88099-6e9f-4b32-af14-8907cf6d24bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0467340db5bb49ed83e8788b24ef2356, user_id=231a7e71361648a3a90c63534986b618, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:43.773432 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb88099-6e9f-4b32-af14-8907cf6d24bd admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:43.844740 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1394/2813] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:13:43 2022] PUT /identity/v3/projects/0467340db5bb49ed83e8788b24ef2356/users/231a7e71361648a3a90c63534986b618/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:13:43.848682 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc3fbf-ce83-4157-826c-709247c42f58 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:43.858397 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc3fbf-ce83-4157-826c-709247c42f58 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:43.916259 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6bc00-ae0b-4b8c-8274-279af770e914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:43.916858 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6bc00-ae0b-4b8c-8274-279af770e914 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:43.917134 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6bc00-ae0b-4b8c-8274-279af770e914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:43.917424 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6bc00-ae0b-4b8c-8274-279af770e914 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:43.918091 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c6bc00-ae0b-4b8c-8274-279af770e914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:43.919704 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c6bc00-ae0b-4b8c-8274-279af770e914 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:43.930187 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1420/2814] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:13:43.934451 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb88099-6e9f-4b32-af14-8907cf6d24bd admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:43.945958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb88099-6e9f-4b32-af14-8907cf6d24bd admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:43.950495 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f3742-4727-4096-8fc7-b393065886b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:43.951170 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f3742-4727-4096-8fc7-b393065886b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:43.951473 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f3742-4727-4096-8fc7-b393065886b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:43.951746 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f3742-4727-4096-8fc7-b393065886b2 admin admin] PATH_INFO: `/v3/projects/0467340db5bb49ed83e8788b24ef2356/users/231a7e71361648a3a90c63534986b618/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:43.953670 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f3742-4727-4096-8fc7-b393065886b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0467340db5bb49ed83e8788b24ef2356, user_id=231a7e71361648a3a90c63534986b618, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:43.956644 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f3742-4727-4096-8fc7-b393065886b2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:43.967202 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1395/2815] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:13:43 2022] PUT /identity/v3/projects/0467340db5bb49ed83e8788b24ef2356/users/231a7e71361648a3a90c63534986b618/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:13:43.978184 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba656b8-b5e1-4b0a-932c-8296fbcb42d4 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:43.978466 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba656b8-b5e1-4b0a-932c-8296fbcb42d4 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:43.978782 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba656b8-b5e1-4b0a-932c-8296fbcb42d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:44.007289 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-6ba656b8-b5e1-4b0a-932c-8296fbcb42d4 None None] MFA Rules not processed for user `231a7e71361648a3a90c63534986b618`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:13:44.008488 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-6ba656b8-b5e1-4b0a-932c-8296fbcb42d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:44.020898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:13:44.079161 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1421/2816] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:13:43 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:13:44.090720 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f3742-4727-4096-8fc7-b393065886b2 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:44.092391 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-631f3742-4727-4096-8fc7-b393065886b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:44.150226 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f3742-4727-4096-8fc7-b393065886b2 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:44.153866 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952aa24-addc-494e-b900-cb41ae75e925 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:44.154453 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952aa24-addc-494e-b900-cb41ae75e925 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:44.154764 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952aa24-addc-494e-b900-cb41ae75e925 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:44.155039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952aa24-addc-494e-b900-cb41ae75e925 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:44.164701 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2952aa24-addc-494e-b900-cb41ae75e925 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:44.166127 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2952aa24-addc-494e-b900-cb41ae75e925 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:44.175335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1396/2817] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:13:44 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:13:44.204036 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba656b8-b5e1-4b0a-932c-8296fbcb42d4 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:44.265017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba656b8-b5e1-4b0a-932c-8296fbcb42d4 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:44.269796 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc449fa-99d1-4279-889d-41bada7e38b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:44.270417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc449fa-99d1-4279-889d-41bada7e38b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:44.270739 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc449fa-99d1-4279-889d-41bada7e38b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:44.271014 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc449fa-99d1-4279-889d-41bada7e38b0 admin admin] PATH_INFO: `/v3/users/07353ae77ab54bd9b4e7b6279ae9c071` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:44.272160 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc449fa-99d1-4279-889d-41bada7e38b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07353ae77ab54bd9b4e7b6279ae9c071)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:44.274086 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc449fa-99d1-4279-889d-41bada7e38b0 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:44.493753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-dbc449fa-99d1-4279-889d-41bada7e38b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07353ae77ab54bd9b4e7b6279ae9c071'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:44.506387 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-dbc449fa-99d1-4279-889d-41bada7e38b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07353ae77ab54bd9b4e7b6279ae9c071'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:44.513450 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-dbc449fa-99d1-4279-889d-41bada7e38b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07353ae77ab54bd9b4e7b6279ae9c071'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:44.514146 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-dbc449fa-99d1-4279-889d-41bada7e38b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07353ae77ab54bd9b4e7b6279ae9c071'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:44.582781 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-dbc449fa-99d1-4279-889d-41bada7e38b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07353ae77ab54bd9b4e7b6279ae9c071'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:44.589168 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1422/2818] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:13:44 2022] DELETE /identity/v3/users/07353ae77ab54bd9b4e7b6279ae9c071 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:13:44.593644 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a7ca0-21be-4310-beb2-5c593bc38c85 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:44.593928 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a7ca0-21be-4310-beb2-5c593bc38c85 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:44.594220 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a7ca0-21be-4310-beb2-5c593bc38c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:44.613316 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-715a7ca0-21be-4310-beb2-5c593bc38c85 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:13:44.614524 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-715a7ca0-21be-4310-beb2-5c593bc38c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:44.638600 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:13:44.673992 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1397/2819] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:13:44.684081 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc449fa-99d1-4279-889d-41bada7e38b0 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:44.685639 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-dbc449fa-99d1-4279-889d-41bada7e38b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:44.725499 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:13:44.747984 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc449fa-99d1-4279-889d-41bada7e38b0 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:44.751581 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c02b8-68b2-471e-8ecf-3ed90c69d7f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:44.752185 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c02b8-68b2-471e-8ecf-3ed90c69d7f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:44.752463 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c02b8-68b2-471e-8ecf-3ed90c69d7f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:44.752735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c02b8-68b2-471e-8ecf-3ed90c69d7f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:44.762399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c02b8-68b2-471e-8ecf-3ed90c69d7f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:44.763907 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c02b8-68b2-471e-8ecf-3ed90c69d7f0 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:44.775481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1423/2820] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:13:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:13:45.189292 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a7ca0-21be-4310-beb2-5c593bc38c85 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:45.190838 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-715a7ca0-21be-4310-beb2-5c593bc38c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:45.206821 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a7ca0-21be-4310-beb2-5c593bc38c85 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:45.212834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6898d4-77dc-4dc3-87b5-2d099abcec30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:45.217778 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6898d4-77dc-4dc3-87b5-2d099abcec30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:45.217778 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6898d4-77dc-4dc3-87b5-2d099abcec30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:45.217778 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6898d4-77dc-4dc3-87b5-2d099abcec30 admin admin] PATH_INFO: `/v3/projects/0bcc6935200a4726acaa3f6d64d25043` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:45.217778 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6898d4-77dc-4dc3-87b5-2d099abcec30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bcc6935200a4726acaa3f6d64d25043)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:45.217778 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6898d4-77dc-4dc3-87b5-2d099abcec30 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:45.320776 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4b6898d4-77dc-4dc3-87b5-2d099abcec30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bcc6935200a4726acaa3f6d64d25043', 'request_id': 'req-4b6898d4-77dc-4dc3-87b5-2d099abcec30'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:45.327142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4b6898d4-77dc-4dc3-87b5-2d099abcec30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bcc6935200a4726acaa3f6d64d25043', 'request_id': 'req-4b6898d4-77dc-4dc3-87b5-2d099abcec30'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:45.333185 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4b6898d4-77dc-4dc3-87b5-2d099abcec30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bcc6935200a4726acaa3f6d64d25043', 'request_id': 'req-4b6898d4-77dc-4dc3-87b5-2d099abcec30'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:45.402215 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4b6898d4-77dc-4dc3-87b5-2d099abcec30 admin admin] The token cache is being invalidate because project 0bcc6935200a4726acaa3f6d64d25043 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:13:45.402508 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4b6898d4-77dc-4dc3-87b5-2d099abcec30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:45.403147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-4b6898d4-77dc-4dc3-87b5-2d099abcec30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:45.404673 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1398/2821] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:13:45 2022] DELETE /identity/v3/projects/0bcc6935200a4726acaa3f6d64d25043 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:13:45.408690 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c02b8-68b2-471e-8ecf-3ed90c69d7f0 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:45.418735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c02b8-68b2-471e-8ecf-3ed90c69d7f0 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:45.476548 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644f23b-57b8-4598-b41f-34306cffc5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:45.477161 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644f23b-57b8-4598-b41f-34306cffc5ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:45.477435 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644f23b-57b8-4598-b41f-34306cffc5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:45.477701 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644f23b-57b8-4598-b41f-34306cffc5ce admin admin] PATH_INFO: `/v3/users/231a7e71361648a3a90c63534986b618` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:45.478929 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644f23b-57b8-4598-b41f-34306cffc5ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=231a7e71361648a3a90c63534986b618)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:45.480868 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644f23b-57b8-4598-b41f-34306cffc5ce admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:45.634972 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7644f23b-57b8-4598-b41f-34306cffc5ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '231a7e71361648a3a90c63534986b618'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:45.646732 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7644f23b-57b8-4598-b41f-34306cffc5ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '231a7e71361648a3a90c63534986b618'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:45.653554 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7644f23b-57b8-4598-b41f-34306cffc5ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '231a7e71361648a3a90c63534986b618'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:45.654202 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7644f23b-57b8-4598-b41f-34306cffc5ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '231a7e71361648a3a90c63534986b618'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:45.695915 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-7644f23b-57b8-4598-b41f-34306cffc5ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '231a7e71361648a3a90c63534986b618'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:45.701825 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1424/2822] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:13:45 2022] DELETE /identity/v3/users/231a7e71361648a3a90c63534986b618 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:13:46.097326 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6898d4-77dc-4dc3-87b5-2d099abcec30 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:46.099027 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-4b6898d4-77dc-4dc3-87b5-2d099abcec30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:46.112658 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6898d4-77dc-4dc3-87b5-2d099abcec30 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:46.172282 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b8809-5f73-4921-9da3-6ec433c24df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:46.172915 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b8809-5f73-4921-9da3-6ec433c24df0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:46.173193 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b8809-5f73-4921-9da3-6ec433c24df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:46.173466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b8809-5f73-4921-9da3-6ec433c24df0 admin admin] PATH_INFO: `/v3/projects/0467340db5bb49ed83e8788b24ef2356` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:46.174611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b8809-5f73-4921-9da3-6ec433c24df0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0467340db5bb49ed83e8788b24ef2356)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:46.176636 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b8809-5f73-4921-9da3-6ec433c24df0 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:46.302628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-481b8809-5f73-4921-9da3-6ec433c24df0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0467340db5bb49ed83e8788b24ef2356', 'request_id': 'req-481b8809-5f73-4921-9da3-6ec433c24df0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:46.309089 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-481b8809-5f73-4921-9da3-6ec433c24df0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0467340db5bb49ed83e8788b24ef2356', 'request_id': 'req-481b8809-5f73-4921-9da3-6ec433c24df0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:46.310286 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:13:46.315343 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-481b8809-5f73-4921-9da3-6ec433c24df0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0467340db5bb49ed83e8788b24ef2356', 'request_id': 'req-481b8809-5f73-4921-9da3-6ec433c24df0'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:46.381683 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-481b8809-5f73-4921-9da3-6ec433c24df0 admin admin] The token cache is being invalidate because project 0467340db5bb49ed83e8788b24ef2356 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:13:46.381967 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-481b8809-5f73-4921-9da3-6ec433c24df0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:46.382609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-481b8809-5f73-4921-9da3-6ec433c24df0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:46.384100 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1399/2823] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:13:46 2022] DELETE /identity/v3/projects/0467340db5bb49ed83e8788b24ef2356 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:13:46.389029 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37349cc-2dbc-4724-ae0b-f466019fda96 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:46.389539 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37349cc-2dbc-4724-ae0b-f466019fda96 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:46.389539 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37349cc-2dbc-4724-ae0b-f466019fda96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:46.409837 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b37349cc-2dbc-4724-ae0b-f466019fda96 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:13:46.411111 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b37349cc-2dbc-4724-ae0b-f466019fda96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:46.459088 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:13:46.471247 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1425/2824] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:13:46.479812 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a4944-6ea6-45c7-a9d6-9434c8c47702 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:46.480092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a4944-6ea6-45c7-a9d6-9434c8c47702 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:46.480360 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a4944-6ea6-45c7-a9d6-9434c8c47702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:46.500311 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-be2a4944-6ea6-45c7-a9d6-9434c8c47702 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:13:46.502817 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-be2a4944-6ea6-45c7-a9d6-9434c8c47702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:46.510114 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1400/2825] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:13:46.514353 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37349cc-2dbc-4724-ae0b-f466019fda96 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:46.524149 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37349cc-2dbc-4724-ae0b-f466019fda96 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:46.528753 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e1609f-ac6f-472f-b7a8-9244cbb7306a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:46.529405 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1609f-ac6f-472f-b7a8-9244cbb7306a admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:46.529927 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1609f-ac6f-472f-b7a8-9244cbb7306a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:46.530374 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1609f-ac6f-472f-b7a8-9244cbb7306a admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:46.531317 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e1609f-ac6f-472f-b7a8-9244cbb7306a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:13:46.531654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e1609f-ac6f-472f-b7a8-9244cbb7306a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:46.533293 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e1609f-ac6f-472f-b7a8-9244cbb7306a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:46.545481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1426/2826] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:13:46.549995 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a4944-6ea6-45c7-a9d6-9434c8c47702 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:46.559729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a4944-6ea6-45c7-a9d6-9434c8c47702 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:46.564319 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126b113-2a7a-46e4-95db-75f6146aa1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:46.564896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126b113-2a7a-46e4-95db-75f6146aa1c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:46.565170 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126b113-2a7a-46e4-95db-75f6146aa1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:46.565438 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126b113-2a7a-46e4-95db-75f6146aa1c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:46.566068 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c126b113-2a7a-46e4-95db-75f6146aa1c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:46.568021 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c126b113-2a7a-46e4-95db-75f6146aa1c4 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:46.684588 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1401/2827] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:13:46 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:13:46.688736 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e1609f-ac6f-472f-b7a8-9244cbb7306a admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:46.698781 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e1609f-ac6f-472f-b7a8-9244cbb7306a admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:46.757603 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d597b-bf24-4b13-8f4a-fa80717fb14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:46.758272 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d597b-bf24-4b13-8f4a-fa80717fb14e admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:46.758616 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d597b-bf24-4b13-8f4a-fa80717fb14e admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:46.759136 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d597b-bf24-4b13-8f4a-fa80717fb14e admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:46.759888 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d597b-bf24-4b13-8f4a-fa80717fb14e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:46.762246 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d597b-bf24-4b13-8f4a-fa80717fb14e admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:46.839689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1427/2828] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:13:46 2022] POST /identity/v3/users => generated 293 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:13:46.843814 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126b113-2a7a-46e4-95db-75f6146aa1c4 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:46.855927 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126b113-2a7a-46e4-95db-75f6146aa1c4 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:46.861410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b16c68-eeba-48b1-8ce3-eab31398aaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:46.862125 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b16c68-eeba-48b1-8ce3-eab31398aaa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:46.862466 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b16c68-eeba-48b1-8ce3-eab31398aaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:46.862845 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b16c68-eeba-48b1-8ce3-eab31398aaa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:46.863585 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b16c68-eeba-48b1-8ce3-eab31398aaa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:46.865204 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b16c68-eeba-48b1-8ce3-eab31398aaa6 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:46.875458 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1402/2829] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:13:46.879906 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d597b-bf24-4b13-8f4a-fa80717fb14e admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:46.889991 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d597b-bf24-4b13-8f4a-fa80717fb14e admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:46.895180 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d26a4-0ba5-4d0a-abf8-3694b606be09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:46.895796 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d26a4-0ba5-4d0a-abf8-3694b606be09 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:46.896051 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d26a4-0ba5-4d0a-abf8-3694b606be09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:46.896322 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d26a4-0ba5-4d0a-abf8-3694b606be09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:46.897228 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d26a4-0ba5-4d0a-abf8-3694b606be09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:46.898875 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d26a4-0ba5-4d0a-abf8-3694b606be09 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:46.909399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1428/2830] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:13:46.914157 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b16c68-eeba-48b1-8ce3-eab31398aaa6 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:46.924414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b16c68-eeba-48b1-8ce3-eab31398aaa6 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:46.929414 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1881d8a-f9ee-43c0-a3e5-a0a444019a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:46.930098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1881d8a-f9ee-43c0-a3e5-a0a444019a33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:46.930375 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1881d8a-f9ee-43c0-a3e5-a0a444019a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:46.930649 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1881d8a-f9ee-43c0-a3e5-a0a444019a33 admin admin] PATH_INFO: `/v3/projects/757ea78d80e743d68982d93716a0e432/users/6a43e36b6eb142c39de2ff7f2290968e/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:46.944974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1881d8a-f9ee-43c0-a3e5-a0a444019a33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=757ea78d80e743d68982d93716a0e432, user_id=6a43e36b6eb142c39de2ff7f2290968e, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:46.947684 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1881d8a-f9ee-43c0-a3e5-a0a444019a33 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:47.038332 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1403/2831] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:13:46 2022] PUT /identity/v3/projects/757ea78d80e743d68982d93716a0e432/users/6a43e36b6eb142c39de2ff7f2290968e/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:13:47.050178 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463cb55-59d5-41de-903c-217b5caf66cc None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:47.051974 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463cb55-59d5-41de-903c-217b5caf66cc None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:47.051974 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463cb55-59d5-41de-903c-217b5caf66cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:47.084183 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-7463cb55-59d5-41de-903c-217b5caf66cc None None] MFA Rules not processed for user `6a43e36b6eb142c39de2ff7f2290968e`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:13:47.085389 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-7463cb55-59d5-41de-903c-217b5caf66cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:47.149752 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1429/2832] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:13:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:13:47.154201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1881d8a-f9ee-43c0-a3e5-a0a444019a33 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:47.215110 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1881d8a-f9ee-43c0-a3e5-a0a444019a33 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:47.219866 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e107b3-5e21-4fdd-afa9-ff4d94237f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:47.220459 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e107b3-5e21-4fdd-afa9-ff4d94237f2b admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:47.220742 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e107b3-5e21-4fdd-afa9-ff4d94237f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:47.221015 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e107b3-5e21-4fdd-afa9-ff4d94237f2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:47.221784 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e107b3-5e21-4fdd-afa9-ff4d94237f2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:47.224031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e107b3-5e21-4fdd-afa9-ff4d94237f2b admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:47.320808 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1404/2833] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:13:47 2022] POST /identity/v3/projects => generated 385 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:13:47.325074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463cb55-59d5-41de-903c-217b5caf66cc None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:47.336430 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463cb55-59d5-41de-903c-217b5caf66cc None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:47.382731 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:13:47.409806 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92cc11-7be0-4198-af38-521f334fbde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:47.409806 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92cc11-7be0-4198-af38-521f334fbde6 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:47.409806 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92cc11-7be0-4198-af38-521f334fbde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:47.409806 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92cc11-7be0-4198-af38-521f334fbde6 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:47.411474 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b92cc11-7be0-4198-af38-521f334fbde6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:47.411474 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b92cc11-7be0-4198-af38-521f334fbde6 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:47.604810 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1430/2834] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:13:47 2022] POST /identity/v3/users => generated 299 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:13:47.608769 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e107b3-5e21-4fdd-afa9-ff4d94237f2b admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:47.619534 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e107b3-5e21-4fdd-afa9-ff4d94237f2b admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:47.624153 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c509cc-7eb5-41e1-a1ab-239c4d3b82c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:47.628531 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c509cc-7eb5-41e1-a1ab-239c4d3b82c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:47.628531 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c509cc-7eb5-41e1-a1ab-239c4d3b82c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:47.628531 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c509cc-7eb5-41e1-a1ab-239c4d3b82c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:47.628531 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c509cc-7eb5-41e1-a1ab-239c4d3b82c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:47.628531 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c509cc-7eb5-41e1-a1ab-239c4d3b82c1 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:47.639048 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1405/2835] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:13:47.643209 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92cc11-7be0-4198-af38-521f334fbde6 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:47.653267 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92cc11-7be0-4198-af38-521f334fbde6 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:47.657779 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e40e7-b956-4406-ba56-12e18dc6a033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:47.658421 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e40e7-b956-4406-ba56-12e18dc6a033 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:47.658730 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e40e7-b956-4406-ba56-12e18dc6a033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:47.659047 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e40e7-b956-4406-ba56-12e18dc6a033 admin admin] PATH_INFO: `/v3/projects/561cf99dde694bfd95804f6c5a3997e0/users/afff20ca4f1b4696a466ae2cefa020f4/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:47.673798 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e40e7-b956-4406-ba56-12e18dc6a033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=561cf99dde694bfd95804f6c5a3997e0, user_id=afff20ca4f1b4696a466ae2cefa020f4, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:47.676987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e40e7-b956-4406-ba56-12e18dc6a033 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:47.743147 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1431/2836] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:13:47 2022] PUT /identity/v3/projects/561cf99dde694bfd95804f6c5a3997e0/users/afff20ca4f1b4696a466ae2cefa020f4/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:13:47.747115 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c509cc-7eb5-41e1-a1ab-239c4d3b82c1 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:47.757234 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c509cc-7eb5-41e1-a1ab-239c4d3b82c1 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:47.806471 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:13:47.815397 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63e5c7-7090-4c35-9977-e0c6de17083a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:47.820278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63e5c7-7090-4c35-9977-e0c6de17083a admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:47.820278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63e5c7-7090-4c35-9977-e0c6de17083a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:47.820278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63e5c7-7090-4c35-9977-e0c6de17083a admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:47.820278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce63e5c7-7090-4c35-9977-e0c6de17083a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:47.820278 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce63e5c7-7090-4c35-9977-e0c6de17083a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:47.828826 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1406/2837] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:13:47.833491 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e40e7-b956-4406-ba56-12e18dc6a033 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:47.843997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e40e7-b956-4406-ba56-12e18dc6a033 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:47.849164 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21f5179-cb4d-466c-9947-9a9984ea26ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:47.849809 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f5179-cb4d-466c-9947-9a9984ea26ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:47.850085 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f5179-cb4d-466c-9947-9a9984ea26ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:47.850384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f5179-cb4d-466c-9947-9a9984ea26ad admin admin] PATH_INFO: `/v3/projects/561cf99dde694bfd95804f6c5a3997e0/users/afff20ca4f1b4696a466ae2cefa020f4/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:47.852373 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21f5179-cb4d-466c-9947-9a9984ea26ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=561cf99dde694bfd95804f6c5a3997e0, user_id=afff20ca4f1b4696a466ae2cefa020f4, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:47.855116 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21f5179-cb4d-466c-9947-9a9984ea26ad admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:47.960160 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1432/2838] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:13:47 2022] PUT /identity/v3/projects/561cf99dde694bfd95804f6c5a3997e0/users/afff20ca4f1b4696a466ae2cefa020f4/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:13:47.970296 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ba628-05ac-4ae9-8e0d-fc2d405ab643 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:47.970579 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ba628-05ac-4ae9-8e0d-fc2d405ab643 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:47.970882 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ba628-05ac-4ae9-8e0d-fc2d405ab643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:47.999182 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-d18ba628-05ac-4ae9-8e0d-fc2d405ab643 None None] MFA Rules not processed for user `afff20ca4f1b4696a466ae2cefa020f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:13:48.000373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-d18ba628-05ac-4ae9-8e0d-fc2d405ab643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:48.076828 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1407/2839] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:13:47 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:13:48.088427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21f5179-cb4d-466c-9947-9a9984ea26ad admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:48.090105 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c21f5179-cb4d-466c-9947-9a9984ea26ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:48.147388 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21f5179-cb4d-466c-9947-9a9984ea26ad admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:48.150931 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d69ab-d0e1-4c92-9055-5a3705da0a64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:48.151531 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d69ab-d0e1-4c92-9055-5a3705da0a64 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:48.151807 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d69ab-d0e1-4c92-9055-5a3705da0a64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:48.152074 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d69ab-d0e1-4c92-9055-5a3705da0a64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:48.161687 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d69ab-d0e1-4c92-9055-5a3705da0a64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:48.163176 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d69ab-d0e1-4c92-9055-5a3705da0a64 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:48.172427 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1433/2840] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:13:48 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:13:49.345218 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ba628-05ac-4ae9-8e0d-fc2d405ab643 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:49.408912 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ba628-05ac-4ae9-8e0d-fc2d405ab643 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:49.413928 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f0537-1c98-4905-a2f4-4534e2b6f3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:49.414551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f0537-1c98-4905-a2f4-4534e2b6f3d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:49.414856 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f0537-1c98-4905-a2f4-4534e2b6f3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:49.415129 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f0537-1c98-4905-a2f4-4534e2b6f3d8 admin admin] PATH_INFO: `/v3/users/6a43e36b6eb142c39de2ff7f2290968e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:49.416253 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f0537-1c98-4905-a2f4-4534e2b6f3d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a43e36b6eb142c39de2ff7f2290968e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:49.418215 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f0537-1c98-4905-a2f4-4534e2b6f3d8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:49.605072 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-437f0537-1c98-4905-a2f4-4534e2b6f3d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a43e36b6eb142c39de2ff7f2290968e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:49.611837 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-437f0537-1c98-4905-a2f4-4534e2b6f3d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a43e36b6eb142c39de2ff7f2290968e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:49.624180 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-437f0537-1c98-4905-a2f4-4534e2b6f3d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a43e36b6eb142c39de2ff7f2290968e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:49.626708 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:13:49.728560 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-437f0537-1c98-4905-a2f4-4534e2b6f3d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a43e36b6eb142c39de2ff7f2290968e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:49.729318 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-437f0537-1c98-4905-a2f4-4534e2b6f3d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a43e36b6eb142c39de2ff7f2290968e'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:49.735778 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1408/2841] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:13:49 2022] DELETE /identity/v3/users/6a43e36b6eb142c39de2ff7f2290968e => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:13:49.740108 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5d6e8-371f-4c3f-b66e-dbc860241342 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:49.740599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5d6e8-371f-4c3f-b66e-dbc860241342 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:49.740729 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5d6e8-371f-4c3f-b66e-dbc860241342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:49.760987 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b2f5d6e8-371f-4c3f-b66e-dbc860241342 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:13:49.766902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b2f5d6e8-371f-4c3f-b66e-dbc860241342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:49.820481 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1434/2842] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:13:49.830399 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f0537-1c98-4905-a2f4-4534e2b6f3d8 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:49.832152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-437f0537-1c98-4905-a2f4-4534e2b6f3d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:49.888729 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f0537-1c98-4905-a2f4-4534e2b6f3d8 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:49.892230 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40af4d-ff3a-4483-86c8-6389b68cfb86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:49.892829 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40af4d-ff3a-4483-86c8-6389b68cfb86 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:49.893101 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40af4d-ff3a-4483-86c8-6389b68cfb86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:49.893367 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40af4d-ff3a-4483-86c8-6389b68cfb86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:49.902324 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed40af4d-ff3a-4483-86c8-6389b68cfb86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:49.903790 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed40af4d-ff3a-4483-86c8-6389b68cfb86 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:49.912902 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1409/2843] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:13:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:13:50.326526 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5d6e8-371f-4c3f-b66e-dbc860241342 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:50.328005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b2f5d6e8-371f-4c3f-b66e-dbc860241342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:50.341197 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5d6e8-371f-4c3f-b66e-dbc860241342 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:50.346158 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609cb74a-4b93-48f2-a6a7-7f5b707d66c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:50.346821 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cb74a-4b93-48f2-a6a7-7f5b707d66c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:50.347100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cb74a-4b93-48f2-a6a7-7f5b707d66c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:50.347372 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cb74a-4b93-48f2-a6a7-7f5b707d66c3 admin admin] PATH_INFO: `/v3/projects/757ea78d80e743d68982d93716a0e432` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:50.348626 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609cb74a-4b93-48f2-a6a7-7f5b707d66c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=757ea78d80e743d68982d93716a0e432)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:50.350650 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609cb74a-4b93-48f2-a6a7-7f5b707d66c3 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:50.454585 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:13:50.462163 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-609cb74a-4b93-48f2-a6a7-7f5b707d66c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '757ea78d80e743d68982d93716a0e432', 'request_id': 'req-609cb74a-4b93-48f2-a6a7-7f5b707d66c3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:50.533896 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-609cb74a-4b93-48f2-a6a7-7f5b707d66c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '757ea78d80e743d68982d93716a0e432', 'request_id': 'req-609cb74a-4b93-48f2-a6a7-7f5b707d66c3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:50.540325 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-609cb74a-4b93-48f2-a6a7-7f5b707d66c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '757ea78d80e743d68982d93716a0e432', 'request_id': 'req-609cb74a-4b93-48f2-a6a7-7f5b707d66c3'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:50.551682 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-609cb74a-4b93-48f2-a6a7-7f5b707d66c3 admin admin] The token cache is being invalidate because project 757ea78d80e743d68982d93716a0e432 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:13:50.551967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-609cb74a-4b93-48f2-a6a7-7f5b707d66c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:50.552599 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-609cb74a-4b93-48f2-a6a7-7f5b707d66c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:50.554102 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1435/2844] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:13:50 2022] DELETE /identity/v3/projects/757ea78d80e743d68982d93716a0e432 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:13:50.558069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40af4d-ff3a-4483-86c8-6389b68cfb86 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:50.567826 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40af4d-ff3a-4483-86c8-6389b68cfb86 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:50.623254 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff17cb-1abc-4b35-b68a-a0619adffdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:50.623875 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff17cb-1abc-4b35-b68a-a0619adffdc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:50.624148 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff17cb-1abc-4b35-b68a-a0619adffdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:50.624412 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff17cb-1abc-4b35-b68a-a0619adffdc3 admin admin] PATH_INFO: `/v3/users/afff20ca4f1b4696a466ae2cefa020f4` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:50.625508 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff17cb-1abc-4b35-b68a-a0619adffdc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afff20ca4f1b4696a466ae2cefa020f4)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:50.627453 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff17cb-1abc-4b35-b68a-a0619adffdc3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:50.754809 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b8ff17cb-1abc-4b35-b68a-a0619adffdc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afff20ca4f1b4696a466ae2cefa020f4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:50.762047 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b8ff17cb-1abc-4b35-b68a-a0619adffdc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afff20ca4f1b4696a466ae2cefa020f4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:50.774361 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b8ff17cb-1abc-4b35-b68a-a0619adffdc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afff20ca4f1b4696a466ae2cefa020f4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:50.827364 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b8ff17cb-1abc-4b35-b68a-a0619adffdc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afff20ca4f1b4696a466ae2cefa020f4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:50.828043 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-b8ff17cb-1abc-4b35-b68a-a0619adffdc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afff20ca4f1b4696a466ae2cefa020f4'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:50.834353 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1410/2845] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:13:50 2022] DELETE /identity/v3/users/afff20ca4f1b4696a466ae2cefa020f4 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:13:51.355062 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609cb74a-4b93-48f2-a6a7-7f5b707d66c3 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:51.361424 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-609cb74a-4b93-48f2-a6a7-7f5b707d66c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:51.370284 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609cb74a-4b93-48f2-a6a7-7f5b707d66c3 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:51.426898 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83fe359-3b4f-487e-bab3-48a5ace29d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:51.427520 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fe359-3b4f-487e-bab3-48a5ace29d7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:51.427793 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fe359-3b4f-487e-bab3-48a5ace29d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:51.428057 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fe359-3b4f-487e-bab3-48a5ace29d7a admin admin] PATH_INFO: `/v3/projects/561cf99dde694bfd95804f6c5a3997e0` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:51.429165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83fe359-3b4f-487e-bab3-48a5ace29d7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=561cf99dde694bfd95804f6c5a3997e0)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:51.431763 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83fe359-3b4f-487e-bab3-48a5ace29d7a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:51.546871 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f83fe359-3b4f-487e-bab3-48a5ace29d7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '561cf99dde694bfd95804f6c5a3997e0', 'request_id': 'req-f83fe359-3b4f-487e-bab3-48a5ace29d7a'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:51.631779 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f83fe359-3b4f-487e-bab3-48a5ace29d7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '561cf99dde694bfd95804f6c5a3997e0', 'request_id': 'req-f83fe359-3b4f-487e-bab3-48a5ace29d7a'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:51.638005 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f83fe359-3b4f-487e-bab3-48a5ace29d7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '561cf99dde694bfd95804f6c5a3997e0', 'request_id': 'req-f83fe359-3b4f-487e-bab3-48a5ace29d7a'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:51.649746 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f83fe359-3b4f-487e-bab3-48a5ace29d7a admin admin] The token cache is being invalidate because project 561cf99dde694bfd95804f6c5a3997e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:13:51.650030 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f83fe359-3b4f-487e-bab3-48a5ace29d7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:51.650693 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-f83fe359-3b4f-487e-bab3-48a5ace29d7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:51.652163 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1436/2846] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:13:51 2022] DELETE /identity/v3/projects/561cf99dde694bfd95804f6c5a3997e0 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:13:51.657401 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7210e81-3c27-46b6-b203-ee0d9716a904 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:51.657830 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7210e81-3c27-46b6-b203-ee0d9716a904 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:51.657930 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7210e81-3c27-46b6-b203-ee0d9716a904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:51.677710 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b7210e81-3c27-46b6-b203-ee0d9716a904 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:13:51.678940 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b7210e81-3c27-46b6-b203-ee0d9716a904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:51.688900 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:13:51.737767 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1411/2847] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:13:51.746368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d4db5-03ce-4f51-930d-07241122dfc7 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:51.750470 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d4db5-03ce-4f51-930d-07241122dfc7 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:51.750470 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d4db5-03ce-4f51-930d-07241122dfc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:51.766585 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-852d4db5-03ce-4f51-930d-07241122dfc7 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:13:51.767802 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-852d4db5-03ce-4f51-930d-07241122dfc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:51.776159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1437/2848] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:13:51.780497 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7210e81-3c27-46b6-b203-ee0d9716a904 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:51.790154 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7210e81-3c27-46b6-b203-ee0d9716a904 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:51.795621 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325425d-9cb8-4172-9954-f03efb87e96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:51.802137 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325425d-9cb8-4172-9954-f03efb87e96f admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:51.802137 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325425d-9cb8-4172-9954-f03efb87e96f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:51.802137 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325425d-9cb8-4172-9954-f03efb87e96f admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:51.802137 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325425d-9cb8-4172-9954-f03efb87e96f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:13:51.802137 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325425d-9cb8-4172-9954-f03efb87e96f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:51.802137 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325425d-9cb8-4172-9954-f03efb87e96f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:51.811121 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1412/2849] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:13:51.815325 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d4db5-03ce-4f51-930d-07241122dfc7 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:51.825180 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d4db5-03ce-4f51-930d-07241122dfc7 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:51.830416 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2458a7c-ebd0-403e-95ff-15efd5fde0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:51.831049 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2458a7c-ebd0-403e-95ff-15efd5fde0a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:51.831322 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2458a7c-ebd0-403e-95ff-15efd5fde0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:51.831910 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2458a7c-ebd0-403e-95ff-15efd5fde0a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:51.832533 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2458a7c-ebd0-403e-95ff-15efd5fde0a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:51.834446 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2458a7c-ebd0-403e-95ff-15efd5fde0a8 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:51.946393 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1438/2850] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:13:51 2022] POST /identity/v3/projects => generated 381 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:13:51.950713 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325425d-9cb8-4172-9954-f03efb87e96f admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:51.961152 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325425d-9cb8-4172-9954-f03efb87e96f admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:52.016086 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c9a6a-8faa-4039-9ea0-f6b4d1ada18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:52.016678 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c9a6a-8faa-4039-9ea0-f6b4d1ada18e admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:52.017555 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c9a6a-8faa-4039-9ea0-f6b4d1ada18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:52.017829 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c9a6a-8faa-4039-9ea0-f6b4d1ada18e admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:52.018456 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c9a6a-8faa-4039-9ea0-f6b4d1ada18e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:52.020384 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c9a6a-8faa-4039-9ea0-f6b4d1ada18e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:52.111734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1413/2851] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:13:51 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:13:52.115522 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2458a7c-ebd0-403e-95ff-15efd5fde0a8 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:52.125997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2458a7c-ebd0-403e-95ff-15efd5fde0a8 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:52.130526 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e4142-a2c6-4a84-b01b-572537ee27d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:52.131157 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e4142-a2c6-4a84-b01b-572537ee27d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:52.131434 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e4142-a2c6-4a84-b01b-572537ee27d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:52.131704 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e4142-a2c6-4a84-b01b-572537ee27d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:52.132337 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e4142-a2c6-4a84-b01b-572537ee27d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:52.133922 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e4142-a2c6-4a84-b01b-572537ee27d4 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:52.144479 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1439/2852] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:13:52.150027 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c9a6a-8faa-4039-9ea0-f6b4d1ada18e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:52.159217 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c9a6a-8faa-4039-9ea0-f6b4d1ada18e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:52.163825 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35faa929-272a-4cf8-840c-4a0b80476bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:52.164406 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35faa929-272a-4cf8-840c-4a0b80476bac admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:52.164684 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35faa929-272a-4cf8-840c-4a0b80476bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:52.164958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35faa929-272a-4cf8-840c-4a0b80476bac admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:52.165592 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35faa929-272a-4cf8-840c-4a0b80476bac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:52.167482 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35faa929-272a-4cf8-840c-4a0b80476bac admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:52.178026 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1414/2853] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:13:52.181925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e4142-a2c6-4a84-b01b-572537ee27d4 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:52.191758 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e4142-a2c6-4a84-b01b-572537ee27d4 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:52.196260 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151d143-2f3d-46f6-bf6c-52299bd5562c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:52.196948 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151d143-2f3d-46f6-bf6c-52299bd5562c admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:52.197159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151d143-2f3d-46f6-bf6c-52299bd5562c admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:52.197424 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151d143-2f3d-46f6-bf6c-52299bd5562c admin admin] PATH_INFO: `/v3/projects/17c91510486c471ab2804cbe3cb1ba66/users/8a0e37a1c09a4c5491361c208cb2a5a8/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:52.212687 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151d143-2f3d-46f6-bf6c-52299bd5562c admin admin] RBAC: Authorizing `identity:create_grant(project_id=17c91510486c471ab2804cbe3cb1ba66, user_id=8a0e37a1c09a4c5491361c208cb2a5a8, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:52.215926 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151d143-2f3d-46f6-bf6c-52299bd5562c admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:52.218230 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:13:52.313261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1440/2854] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:13:52 2022] PUT /identity/v3/projects/17c91510486c471ab2804cbe3cb1ba66/users/8a0e37a1c09a4c5491361c208cb2a5a8/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:13:52.378233 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee6385-3578-4e97-a65a-3178376dbba7 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:52.378508 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee6385-3578-4e97-a65a-3178376dbba7 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:52.378834 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee6385-3578-4e97-a65a-3178376dbba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:52.407124 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-78ee6385-3578-4e97-a65a-3178376dbba7 None None] MFA Rules not processed for user `8a0e37a1c09a4c5491361c208cb2a5a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:13:52.408324 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-78ee6385-3578-4e97-a65a-3178376dbba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:52.472618 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1415/2855] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:13:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:13:52.477112 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151d143-2f3d-46f6-bf6c-52299bd5562c admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:52.538540 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151d143-2f3d-46f6-bf6c-52299bd5562c admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:52.543452 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f96486-662a-4cd3-852c-828ad284f9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:52.544045 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f96486-662a-4cd3-852c-828ad284f9e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:52.544319 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f96486-662a-4cd3-852c-828ad284f9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:52.544708 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f96486-662a-4cd3-852c-828ad284f9e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:52.545335 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f96486-662a-4cd3-852c-828ad284f9e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:52.547254 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f96486-662a-4cd3-852c-828ad284f9e9 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:52.645483 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1441/2856] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:13:52 2022] POST /identity/v3/projects => generated 381 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:13:52.649628 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee6385-3578-4e97-a65a-3178376dbba7 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:52.660430 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee6385-3578-4e97-a65a-3178376dbba7 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:52.668284 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:13:52.719781 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7656a77-bb0b-4828-8f33-930410011d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:52.720418 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7656a77-bb0b-4828-8f33-930410011d2f admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:52.720696 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7656a77-bb0b-4828-8f33-930410011d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:52.720966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7656a77-bb0b-4828-8f33-930410011d2f admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:52.721600 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7656a77-bb0b-4828-8f33-930410011d2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:52.725427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7656a77-bb0b-4828-8f33-930410011d2f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:52.813137 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1416/2857] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:13:52 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:13:52.815829 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f96486-662a-4cd3-852c-828ad284f9e9 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:52.826051 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f96486-662a-4cd3-852c-828ad284f9e9 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:52.830587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012005d-388e-4c41-a0b5-bd3f999abf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:52.836384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012005d-388e-4c41-a0b5-bd3f999abf26 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:52.836384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012005d-388e-4c41-a0b5-bd3f999abf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:52.836384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012005d-388e-4c41-a0b5-bd3f999abf26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:52.836384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012005d-388e-4c41-a0b5-bd3f999abf26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:52.836384 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012005d-388e-4c41-a0b5-bd3f999abf26 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:52.844880 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1442/2858] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:13:52.849016 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7656a77-bb0b-4828-8f33-930410011d2f admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:52.859659 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7656a77-bb0b-4828-8f33-930410011d2f admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:52.864350 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c4254-0612-42fd-b3a9-5d07fbd6de2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:52.864993 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c4254-0612-42fd-b3a9-5d07fbd6de2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:52.865293 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c4254-0612-42fd-b3a9-5d07fbd6de2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:52.865549 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c4254-0612-42fd-b3a9-5d07fbd6de2a admin admin] PATH_INFO: `/v3/projects/d116c4d3dd714e34bd6c47a71dbf1210/users/7d34ec18f3024d1eaf5205d831d168d8/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:52.882263 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c4254-0612-42fd-b3a9-5d07fbd6de2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d116c4d3dd714e34bd6c47a71dbf1210, user_id=7d34ec18f3024d1eaf5205d831d168d8, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:52.884949 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c4254-0612-42fd-b3a9-5d07fbd6de2a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:52.987919 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1417/2859] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:13:52 2022] PUT /identity/v3/projects/d116c4d3dd714e34bd6c47a71dbf1210/users/7d34ec18f3024d1eaf5205d831d168d8/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:13:52.991869 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012005d-388e-4c41-a0b5-bd3f999abf26 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:53.002485 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012005d-388e-4c41-a0b5-bd3f999abf26 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:53.067131 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59444d17-aa57-48ab-bff0-c3218fe13682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:53.067745 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59444d17-aa57-48ab-bff0-c3218fe13682 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:53.068017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59444d17-aa57-48ab-bff0-c3218fe13682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:53.068281 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59444d17-aa57-48ab-bff0-c3218fe13682 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:53.068924 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59444d17-aa57-48ab-bff0-c3218fe13682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:53.070576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59444d17-aa57-48ab-bff0-c3218fe13682 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:53.082856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1443/2860] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:13:53.087047 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c4254-0612-42fd-b3a9-5d07fbd6de2a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:53.097704 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c4254-0612-42fd-b3a9-5d07fbd6de2a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:53.102248 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3fac9-1581-4409-b09a-d1d9ee558109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:53.102920 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3fac9-1581-4409-b09a-d1d9ee558109 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:53.103200 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3fac9-1581-4409-b09a-d1d9ee558109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:53.103472 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3fac9-1581-4409-b09a-d1d9ee558109 admin admin] PATH_INFO: `/v3/projects/d116c4d3dd714e34bd6c47a71dbf1210/users/7d34ec18f3024d1eaf5205d831d168d8/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:53.106147 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3fac9-1581-4409-b09a-d1d9ee558109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d116c4d3dd714e34bd6c47a71dbf1210, user_id=7d34ec18f3024d1eaf5205d831d168d8, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:53.108811 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3fac9-1581-4409-b09a-d1d9ee558109 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:53.118786 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1418/2861] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:13:53 2022] PUT /identity/v3/projects/d116c4d3dd714e34bd6c47a71dbf1210/users/7d34ec18f3024d1eaf5205d831d168d8/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:13:53.140406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75346643-b1e6-4e1f-a962-6950bacb17b2 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:53.140695 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75346643-b1e6-4e1f-a962-6950bacb17b2 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:53.140977 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75346643-b1e6-4e1f-a962-6950bacb17b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:53.169094 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-75346643-b1e6-4e1f-a962-6950bacb17b2 None None] MFA Rules not processed for user `7d34ec18f3024d1eaf5205d831d168d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:13:53.170307 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-75346643-b1e6-4e1f-a962-6950bacb17b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:53.219906 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:13:53.238652 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1444/2862] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:13:53 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:13:53.250153 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3fac9-1581-4409-b09a-d1d9ee558109 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:53.254348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-fcd3fac9-1581-4409-b09a-d1d9ee558109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:53.309625 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3fac9-1581-4409-b09a-d1d9ee558109 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:53.313302 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0354a46d-5235-4674-9c75-07ab82bfe3f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:53.313904 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354a46d-5235-4674-9c75-07ab82bfe3f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:53.314182 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354a46d-5235-4674-9c75-07ab82bfe3f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:53.314453 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354a46d-5235-4674-9c75-07ab82bfe3f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:53.323488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0354a46d-5235-4674-9c75-07ab82bfe3f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:53.324915 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0354a46d-5235-4674-9c75-07ab82bfe3f1 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:53.334373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1419/2863] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:13:53 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:13:53.464680 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75346643-b1e6-4e1f-a962-6950bacb17b2 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:53.474081 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75346643-b1e6-4e1f-a962-6950bacb17b2 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:53.477526 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d96a51-22a9-4f20-8059-e20fbf4ff641 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:53.478116 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d96a51-22a9-4f20-8059-e20fbf4ff641 service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:53.478371 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d96a51-22a9-4f20-8059-e20fbf4ff641 service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:53.478634 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d96a51-22a9-4f20-8059-e20fbf4ff641 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:53.533917 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d96a51-22a9-4f20-8059-e20fbf4ff641 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:53.535278 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d96a51-22a9-4f20-8059-e20fbf4ff641 service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:53.544496 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1445/2864] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:13:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:13:54.000762 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0354a46d-5235-4674-9c75-07ab82bfe3f1 service cinder] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:54.062923 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0354a46d-5235-4674-9c75-07ab82bfe3f1 service cinder] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:54.067388 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7cb4ed-2325-4a21-89fd-939a6aeefcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:54.067996 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cb4ed-2325-4a21-89fd-939a6aeefcff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:54.068287 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cb4ed-2325-4a21-89fd-939a6aeefcff admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:54.068535 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cb4ed-2325-4a21-89fd-939a6aeefcff admin admin] PATH_INFO: `/v3/users/8a0e37a1c09a4c5491361c208cb2a5a8` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:54.069621 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7cb4ed-2325-4a21-89fd-939a6aeefcff admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a0e37a1c09a4c5491361c208cb2a5a8)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:54.071550 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7cb4ed-2325-4a21-89fd-939a6aeefcff admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:54.077310 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:13:54.291707 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cb7cb4ed-2325-4a21-89fd-939a6aeefcff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a0e37a1c09a4c5491361c208cb2a5a8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:54.299081 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cb7cb4ed-2325-4a21-89fd-939a6aeefcff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a0e37a1c09a4c5491361c208cb2a5a8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:54.312175 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cb7cb4ed-2325-4a21-89fd-939a6aeefcff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a0e37a1c09a4c5491361c208cb2a5a8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:54.398989 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cb7cb4ed-2325-4a21-89fd-939a6aeefcff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a0e37a1c09a4c5491361c208cb2a5a8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:54.399637 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-cb7cb4ed-2325-4a21-89fd-939a6aeefcff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a0e37a1c09a4c5491361c208cb2a5a8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:54.406074 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1420/2865] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:13:53 2022] DELETE /identity/v3/users/8a0e37a1c09a4c5491361c208cb2a5a8 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:13:54.410591 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1649d80-e06c-4172-a3a6-d87c22019801 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:54.413794 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1649d80-e06c-4172-a3a6-d87c22019801 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:54.413794 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1649d80-e06c-4172-a3a6-d87c22019801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:54.430770 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-b1649d80-e06c-4172-a3a6-d87c22019801 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:13:54.431967 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b1649d80-e06c-4172-a3a6-d87c22019801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:54.490681 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1446/2866] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:13:54.501563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7cb4ed-2325-4a21-89fd-939a6aeefcff admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:54.503034 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-cb7cb4ed-2325-4a21-89fd-939a6aeefcff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:54.559587 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7cb4ed-2325-4a21-89fd-939a6aeefcff admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:54.563186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ed45f2-5b5d-430e-b058-9ac189bf27e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:54.563781 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed45f2-5b5d-430e-b058-9ac189bf27e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:54.564057 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed45f2-5b5d-430e-b058-9ac189bf27e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:54.564326 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed45f2-5b5d-430e-b058-9ac189bf27e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:54.573457 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ed45f2-5b5d-430e-b058-9ac189bf27e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:54.574965 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ed45f2-5b5d-430e-b058-9ac189bf27e2 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:54.584514 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1421/2867] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:13:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:13:55.012548 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1649d80-e06c-4172-a3a6-d87c22019801 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:55.012548 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-b1649d80-e06c-4172-a3a6-d87c22019801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:55.033765 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1649d80-e06c-4172-a3a6-d87c22019801 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:55.041604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba8d617-3b88-4255-a1fa-48521bad8853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:55.041604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8d617-3b88-4255-a1fa-48521bad8853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:55.041604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8d617-3b88-4255-a1fa-48521bad8853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:55.041604 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8d617-3b88-4255-a1fa-48521bad8853 admin admin] PATH_INFO: `/v3/projects/17c91510486c471ab2804cbe3cb1ba66` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:55.042840 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba8d617-3b88-4255-a1fa-48521bad8853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17c91510486c471ab2804cbe3cb1ba66)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:55.043712 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba8d617-3b88-4255-a1fa-48521bad8853 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:55.146268 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1ba8d617-3b88-4255-a1fa-48521bad8853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17c91510486c471ab2804cbe3cb1ba66', 'request_id': 'req-1ba8d617-3b88-4255-a1fa-48521bad8853'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:55.221548 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1ba8d617-3b88-4255-a1fa-48521bad8853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17c91510486c471ab2804cbe3cb1ba66', 'request_id': 'req-1ba8d617-3b88-4255-a1fa-48521bad8853'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:55.227671 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1ba8d617-3b88-4255-a1fa-48521bad8853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17c91510486c471ab2804cbe3cb1ba66', 'request_id': 'req-1ba8d617-3b88-4255-a1fa-48521bad8853'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:55.238889 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1ba8d617-3b88-4255-a1fa-48521bad8853 admin admin] The token cache is being invalidate because project 17c91510486c471ab2804cbe3cb1ba66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:13:55.239172 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1ba8d617-3b88-4255-a1fa-48521bad8853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:55.239789 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1ba8d617-3b88-4255-a1fa-48521bad8853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:55.241362 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1447/2868] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:13:55 2022] DELETE /identity/v3/projects/17c91510486c471ab2804cbe3cb1ba66 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:13:55.246172 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ed45f2-5b5d-430e-b058-9ac189bf27e2 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:55.255863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ed45f2-5b5d-430e-b058-9ac189bf27e2 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:55.314532 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75537ada-cc4e-4b89-843a-1dd3eb7a7a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:55.315183 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75537ada-cc4e-4b89-843a-1dd3eb7a7a7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:55.315480 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75537ada-cc4e-4b89-843a-1dd3eb7a7a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:55.315765 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75537ada-cc4e-4b89-843a-1dd3eb7a7a7e admin admin] PATH_INFO: `/v3/users/7d34ec18f3024d1eaf5205d831d168d8` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:55.317028 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75537ada-cc4e-4b89-843a-1dd3eb7a7a7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d34ec18f3024d1eaf5205d831d168d8)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:55.319043 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75537ada-cc4e-4b89-843a-1dd3eb7a7a7e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:55.496989 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:13:55.509811 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-75537ada-cc4e-4b89-843a-1dd3eb7a7a7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d34ec18f3024d1eaf5205d831d168d8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:55.516884 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-75537ada-cc4e-4b89-843a-1dd3eb7a7a7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d34ec18f3024d1eaf5205d831d168d8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:55.529705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-75537ada-cc4e-4b89-843a-1dd3eb7a7a7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d34ec18f3024d1eaf5205d831d168d8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:55.627716 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-75537ada-cc4e-4b89-843a-1dd3eb7a7a7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d34ec18f3024d1eaf5205d831d168d8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:55.628348 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-75537ada-cc4e-4b89-843a-1dd3eb7a7a7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d34ec18f3024d1eaf5205d831d168d8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:55.634677 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1422/2869] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:13:55 2022] DELETE /identity/v3/users/7d34ec18f3024d1eaf5205d831d168d8 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:13:56.023352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba8d617-3b88-4255-a1fa-48521bad8853 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:56.030694 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1ba8d617-3b88-4255-a1fa-48521bad8853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:56.038881 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba8d617-3b88-4255-a1fa-48521bad8853 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:56.049488 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:13:56.099306 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f44567-1c3a-4f38-9857-b733f92abb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:56.099925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f44567-1c3a-4f38-9857-b733f92abb7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:56.100195 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f44567-1c3a-4f38-9857-b733f92abb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:56.100460 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f44567-1c3a-4f38-9857-b733f92abb7c admin admin] PATH_INFO: `/v3/projects/d116c4d3dd714e34bd6c47a71dbf1210` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:56.101553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f44567-1c3a-4f38-9857-b733f92abb7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d116c4d3dd714e34bd6c47a71dbf1210)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:56.103571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f44567-1c3a-4f38-9857-b733f92abb7c admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:56.215050 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-50f44567-1c3a-4f38-9857-b733f92abb7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd116c4d3dd714e34bd6c47a71dbf1210', 'request_id': 'req-50f44567-1c3a-4f38-9857-b733f92abb7c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:56.295617 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-50f44567-1c3a-4f38-9857-b733f92abb7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd116c4d3dd714e34bd6c47a71dbf1210', 'request_id': 'req-50f44567-1c3a-4f38-9857-b733f92abb7c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:56.301847 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-50f44567-1c3a-4f38-9857-b733f92abb7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd116c4d3dd714e34bd6c47a71dbf1210', 'request_id': 'req-50f44567-1c3a-4f38-9857-b733f92abb7c'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:56.313245 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-50f44567-1c3a-4f38-9857-b733f92abb7c admin admin] The token cache is being invalidate because project d116c4d3dd714e34bd6c47a71dbf1210 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:13:56.313530 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-50f44567-1c3a-4f38-9857-b733f92abb7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:56.314187 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-50f44567-1c3a-4f38-9857-b733f92abb7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:13:56.315689 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1448/2870] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:13:56 2022] DELETE /identity/v3/projects/d116c4d3dd714e34bd6c47a71dbf1210 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:13:56.320672 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46bd0bf-2639-492a-a251-e3428e0b2c31 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:56.321117 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46bd0bf-2639-492a-a251-e3428e0b2c31 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:56.321248 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46bd0bf-2639-492a-a251-e3428e0b2c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:56.341386 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-f46bd0bf-2639-492a-a251-e3428e0b2c31 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:13:56.342564 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f46bd0bf-2639-492a-a251-e3428e0b2c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:56.399150 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1423/2871] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:13:56.408705 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939aa704-6b38-4f50-80d1-3e27827330c2 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:56.408990 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939aa704-6b38-4f50-80d1-3e27827330c2 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:56.409261 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939aa704-6b38-4f50-80d1-3e27827330c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:56.429345 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-939aa704-6b38-4f50-80d1-3e27827330c2 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:13:56.430547 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-939aa704-6b38-4f50-80d1-3e27827330c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:56.438698 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1449/2872] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:13:56.442840 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46bd0bf-2639-492a-a251-e3428e0b2c31 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:56.452642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46bd0bf-2639-492a-a251-e3428e0b2c31 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:56.457334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9f427-3f19-4378-b34f-a0a737b93ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:56.457947 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9f427-3f19-4378-b34f-a0a737b93ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:56.458221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9f427-3f19-4378-b34f-a0a737b93ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:56.458489 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9f427-3f19-4378-b34f-a0a737b93ca3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:56.459155 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9f427-3f19-4378-b34f-a0a737b93ca3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:13:56.459452 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9f427-3f19-4378-b34f-a0a737b93ca3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:56.461039 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9f427-3f19-4378-b34f-a0a737b93ca3 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:56.473058 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1424/2873] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:13:56.477801 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939aa704-6b38-4f50-80d1-3e27827330c2 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:56.488011 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939aa704-6b38-4f50-80d1-3e27827330c2 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:56.492954 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e88209-b8cb-439d-b90b-e954e90fd8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:56.493535 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e88209-b8cb-439d-b90b-e954e90fd8d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:56.493806 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e88209-b8cb-439d-b90b-e954e90fd8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:56.494069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e88209-b8cb-439d-b90b-e954e90fd8d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:56.494790 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e88209-b8cb-439d-b90b-e954e90fd8d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:56.496957 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e88209-b8cb-439d-b90b-e954e90fd8d7 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:56.629287 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1450/2874] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:13:56 2022] POST /identity/v3/projects => generated 375 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:13:56.633455 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9f427-3f19-4378-b34f-a0a737b93ca3 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:56.644597 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9f427-3f19-4378-b34f-a0a737b93ca3 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:56.701186 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a37e5-facc-4ed6-b886-544ef725e258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:56.701832 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a37e5-facc-4ed6-b886-544ef725e258 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:56.702113 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a37e5-facc-4ed6-b886-544ef725e258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:56.702383 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a37e5-facc-4ed6-b886-544ef725e258 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:56.703049 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a37e5-facc-4ed6-b886-544ef725e258 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:56.704978 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a37e5-facc-4ed6-b886-544ef725e258 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:56.775880 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1425/2875] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:13:56 2022] POST /identity/v3/users => generated 288 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:13:56.783343 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e88209-b8cb-439d-b90b-e954e90fd8d7 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:56.793603 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e88209-b8cb-439d-b90b-e954e90fd8d7 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:56.798188 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b82c50-bb56-46fa-9e34-b3b7bfe4b5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:56.798841 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b82c50-bb56-46fa-9e34-b3b7bfe4b5a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:56.799120 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b82c50-bb56-46fa-9e34-b3b7bfe4b5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:56.799394 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b82c50-bb56-46fa-9e34-b3b7bfe4b5a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:56.800031 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b82c50-bb56-46fa-9e34-b3b7bfe4b5a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:56.801624 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b82c50-bb56-46fa-9e34-b3b7bfe4b5a8 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:56.811441 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1451/2876] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:13:56.815298 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a37e5-facc-4ed6-b886-544ef725e258 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:56.825765 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a37e5-facc-4ed6-b886-544ef725e258 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:56.830469 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c421ac8e-de2d-41b3-8912-d119b6714dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:56.831084 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421ac8e-de2d-41b3-8912-d119b6714dd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:56.831769 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421ac8e-de2d-41b3-8912-d119b6714dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:56.832041 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421ac8e-de2d-41b3-8912-d119b6714dd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:56.832682 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c421ac8e-de2d-41b3-8912-d119b6714dd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:56.834267 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c421ac8e-de2d-41b3-8912-d119b6714dd2 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:56.845551 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1426/2877] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:13:56.849900 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b82c50-bb56-46fa-9e34-b3b7bfe4b5a8 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:56.860602 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b82c50-bb56-46fa-9e34-b3b7bfe4b5a8 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:56.865276 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023e19d-6109-40f7-8aae-dc49a255bd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:56.865917 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023e19d-6109-40f7-8aae-dc49a255bd86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:56.866194 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023e19d-6109-40f7-8aae-dc49a255bd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:56.866465 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023e19d-6109-40f7-8aae-dc49a255bd86 admin admin] PATH_INFO: `/v3/projects/faf80adb087a441b9465e92d668fe6ad/users/957e66a343434847a8152d0216ae920d/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:56.881609 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3023e19d-6109-40f7-8aae-dc49a255bd86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faf80adb087a441b9465e92d668fe6ad, user_id=957e66a343434847a8152d0216ae920d, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:56.884305 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3023e19d-6109-40f7-8aae-dc49a255bd86 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:56.957672 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1452/2878] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:13:56 2022] PUT /identity/v3/projects/faf80adb087a441b9465e92d668fe6ad/users/957e66a343434847a8152d0216ae920d/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:13:56.968576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4123d-3122-470f-8606-a51e11495811 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:56.968856 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4123d-3122-470f-8606-a51e11495811 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:56.969128 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4123d-3122-470f-8606-a51e11495811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:56.995871 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:13:56.998877 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-9be4123d-3122-470f-8606-a51e11495811 None None] MFA Rules not processed for user `957e66a343434847a8152d0216ae920d`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:13:57.000116 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9be4123d-3122-470f-8606-a51e11495811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:57.064159 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1427/2879] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:13:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:13:57.068695 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023e19d-6109-40f7-8aae-dc49a255bd86 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:57.110685 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:13:57.130945 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023e19d-6109-40f7-8aae-dc49a255bd86 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:57.135722 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c75c4-a201-4e69-8274-a6047d24bb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:57.136342 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c75c4-a201-4e69-8274-a6047d24bb72 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:57.136677 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c75c4-a201-4e69-8274-a6047d24bb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:57.137112 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c75c4-a201-4e69-8274-a6047d24bb72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:57.137751 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c75c4-a201-4e69-8274-a6047d24bb72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:57.139678 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c75c4-a201-4e69-8274-a6047d24bb72 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:57.239788 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1453/2880] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:13:57 2022] POST /identity/v3/projects => generated 375 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:13:57.243849 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4123d-3122-470f-8606-a51e11495811 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:57.254558 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4123d-3122-470f-8606-a51e11495811 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:57.310314 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c90819-7403-4e9e-b41d-4c2299f00082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:57.310982 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c90819-7403-4e9e-b41d-4c2299f00082 admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:57.311274 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c90819-7403-4e9e-b41d-4c2299f00082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:57.311612 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c90819-7403-4e9e-b41d-4c2299f00082 admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:57.312261 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c90819-7403-4e9e-b41d-4c2299f00082 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:57.314216 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c90819-7403-4e9e-b41d-4c2299f00082 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:57.410073 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1428/2881] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:13:57 2022] POST /identity/v3/users => generated 294 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:13:57.414032 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c75c4-a201-4e69-8274-a6047d24bb72 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:57.424853 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c75c4-a201-4e69-8274-a6047d24bb72 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:57.429430 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d7a249-2525-4cfe-837c-4df418261401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:57.430015 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7a249-2525-4cfe-837c-4df418261401 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:57.430286 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7a249-2525-4cfe-837c-4df418261401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:57.430551 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d7a249-2525-4cfe-837c-4df418261401 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:57.431237 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d7a249-2525-4cfe-837c-4df418261401 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:57.432977 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d7a249-2525-4cfe-837c-4df418261401 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:57.442899 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1454/2882] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:13:57.446967 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c90819-7403-4e9e-b41d-4c2299f00082 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:57.457674 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c90819-7403-4e9e-b41d-4c2299f00082 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:57.462442 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9385ddf5-9646-4233-a131-9500f688b578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:57.463137 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385ddf5-9646-4233-a131-9500f688b578 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:57.463416 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385ddf5-9646-4233-a131-9500f688b578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:57.463684 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385ddf5-9646-4233-a131-9500f688b578 admin admin] PATH_INFO: `/v3/projects/3ed17d4c7dec40afa9f073609c99784d/users/e05060b24cf542b4ae56f93cd340d9ed/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:57.480042 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9385ddf5-9646-4233-a131-9500f688b578 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ed17d4c7dec40afa9f073609c99784d, user_id=e05060b24cf542b4ae56f93cd340d9ed, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:57.482874 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9385ddf5-9646-4233-a131-9500f688b578 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:57.567971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1429/2883] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:13:57 2022] PUT /identity/v3/projects/3ed17d4c7dec40afa9f073609c99784d/users/e05060b24cf542b4ae56f93cd340d9ed/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:13:57.572270 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d7a249-2525-4cfe-837c-4df418261401 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:57.588266 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d7a249-2525-4cfe-837c-4df418261401 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:57.666116 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add2d15-85f6-49b7-8959-2d13d211a06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:57.666999 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add2d15-85f6-49b7-8959-2d13d211a06d admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:57.667272 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add2d15-85f6-49b7-8959-2d13d211a06d admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:57.668173 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add2d15-85f6-49b7-8959-2d13d211a06d admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:57.668988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add2d15-85f6-49b7-8959-2d13d211a06d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:57.670611 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add2d15-85f6-49b7-8959-2d13d211a06d admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:57.680877 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1455/2884] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:13:57.685774 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9385ddf5-9646-4233-a131-9500f688b578 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:57.695971 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9385ddf5-9646-4233-a131-9500f688b578 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:57.701428 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a121289-6518-46e3-bbc9-25790eb29dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:57.702068 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a121289-6518-46e3-bbc9-25790eb29dfa admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:57.702342 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a121289-6518-46e3-bbc9-25790eb29dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:57.702609 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a121289-6518-46e3-bbc9-25790eb29dfa admin admin] PATH_INFO: `/v3/projects/3ed17d4c7dec40afa9f073609c99784d/users/e05060b24cf542b4ae56f93cd340d9ed/roles/9dce48a2b53c482d931dcd064b07744e` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:57.704539 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a121289-6518-46e3-bbc9-25790eb29dfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ed17d4c7dec40afa9f073609c99784d, user_id=e05060b24cf542b4ae56f93cd340d9ed, role_id=9dce48a2b53c482d931dcd064b07744e)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:57.707454 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a121289-6518-46e3-bbc9-25790eb29dfa admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:57.717478 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1430/2885] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:13:57 2022] PUT /identity/v3/projects/3ed17d4c7dec40afa9f073609c99784d/users/e05060b24cf542b4ae56f93cd340d9ed/roles/9dce48a2b53c482d931dcd064b07744e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:13:57.727473 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af61f20-4bf2-42db-bbad-6302efb4eea8 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:57.727755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af61f20-4bf2-42db-bbad-6302efb4eea8 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:57.728027 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af61f20-4bf2-42db-bbad-6302efb4eea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:57.755648 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-1af61f20-4bf2-42db-bbad-6302efb4eea8 None None] MFA Rules not processed for user `e05060b24cf542b4ae56f93cd340d9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:13:57.756837 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-1af61f20-4bf2-42db-bbad-6302efb4eea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:57.822576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1456/2886] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:13:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:13:57.833954 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a121289-6518-46e3-bbc9-25790eb29dfa admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:13:57.835622 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-5a121289-6518-46e3-bbc9-25790eb29dfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:13:57.892363 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a121289-6518-46e3-bbc9-25790eb29dfa admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:13:57.896018 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8e281-c766-4cb3-a8d2-39bb4e4d3029 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:13:57.896608 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8e281-c766-4cb3-a8d2-39bb4e4d3029 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:13:57.896879 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8e281-c766-4cb3-a8d2-39bb4e4d3029 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:13:57.897143 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8e281-c766-4cb3-a8d2-39bb4e4d3029 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:13:57.905370 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:13:57.952898 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c8e281-c766-4cb3-a8d2-39bb4e4d3029 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:13:57.954272 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c8e281-c766-4cb3-a8d2-39bb4e4d3029 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:13:57.963456 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1431/2887] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:13:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:14:01.053646 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af61f20-4bf2-42db-bbad-6302efb4eea8 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:14:01.089673 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:14:01.116984 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af61f20-4bf2-42db-bbad-6302efb4eea8 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:14:01.121388 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a7e70-b421-4d20-9ebf-5a3f13ede555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:14:01.122011 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a7e70-b421-4d20-9ebf-5a3f13ede555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:14:01.122283 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a7e70-b421-4d20-9ebf-5a3f13ede555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:14:01.122554 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a7e70-b421-4d20-9ebf-5a3f13ede555 admin admin] PATH_INFO: `/v3/users/957e66a343434847a8152d0216ae920d` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:14:01.123744 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a7e70-b421-4d20-9ebf-5a3f13ede555 admin admin] RBAC: Authorizing `identity:delete_user(user_id=957e66a343434847a8152d0216ae920d)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:14:01.125637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a7e70-b421-4d20-9ebf-5a3f13ede555 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:14:01.322315 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-212a7e70-b421-4d20-9ebf-5a3f13ede555 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '957e66a343434847a8152d0216ae920d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:14:01.334560 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-212a7e70-b421-4d20-9ebf-5a3f13ede555 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '957e66a343434847a8152d0216ae920d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:14:01.342030 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-212a7e70-b421-4d20-9ebf-5a3f13ede555 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '957e66a343434847a8152d0216ae920d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:14:01.342737 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-212a7e70-b421-4d20-9ebf-5a3f13ede555 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '957e66a343434847a8152d0216ae920d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:14:01.421258 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-212a7e70-b421-4d20-9ebf-5a3f13ede555 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '957e66a343434847a8152d0216ae920d'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:14:01.427497 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1457/2888] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:14:01 2022] DELETE /identity/v3/users/957e66a343434847a8152d0216ae920d => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:14:01.432054 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b83af-94d7-4f8f-88ae-66d7c0abe63c None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:14:01.435390 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b83af-94d7-4f8f-88ae-66d7c0abe63c None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:14:01.435390 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b83af-94d7-4f8f-88ae-66d7c0abe63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:14:01.452727 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b71b83af-94d7-4f8f-88ae-66d7c0abe63c None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:14:01.454000 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b71b83af-94d7-4f8f-88ae-66d7c0abe63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:14:01.513193 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1432/2889] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:14:01.523069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a7e70-b421-4d20-9ebf-5a3f13ede555 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:14:01.524550 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-212a7e70-b421-4d20-9ebf-5a3f13ede555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:14:01.582283 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a7e70-b421-4d20-9ebf-5a3f13ede555 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:14:01.585836 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b01be6-6c26-4974-abac-af94bcfe7da6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:14:01.586434 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b01be6-6c26-4974-abac-af94bcfe7da6 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:14:01.586735 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b01be6-6c26-4974-abac-af94bcfe7da6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:14:01.587009 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b01be6-6c26-4974-abac-af94bcfe7da6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:14:01.596579 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b01be6-6c26-4974-abac-af94bcfe7da6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:14:01.597992 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b01be6-6c26-4974-abac-af94bcfe7da6 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:14:01.607994 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1458/2890] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:14:02.000231 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b83af-94d7-4f8f-88ae-66d7c0abe63c None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:14:02.001663 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b71b83af-94d7-4f8f-88ae-66d7c0abe63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:14:02.014534 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b83af-94d7-4f8f-88ae-66d7c0abe63c None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:14:02.019234 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc59589-1ce0-4b65-b24d-b30b0faf8965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:14:02.019868 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc59589-1ce0-4b65-b24d-b30b0faf8965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:14:02.020148 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc59589-1ce0-4b65-b24d-b30b0faf8965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:14:02.020401 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc59589-1ce0-4b65-b24d-b30b0faf8965 admin admin] PATH_INFO: `/v3/projects/faf80adb087a441b9465e92d668fe6ad` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:14:02.021689 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc59589-1ce0-4b65-b24d-b30b0faf8965 admin admin] RBAC: Authorizing `identity:delete_project(project_id=faf80adb087a441b9465e92d668fe6ad)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:14:02.023691 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc59589-1ce0-4b65-b24d-b30b0faf8965 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:14:02.121583 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ebc59589-1ce0-4b65-b24d-b30b0faf8965 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faf80adb087a441b9465e92d668fe6ad', 'request_id': 'req-ebc59589-1ce0-4b65-b24d-b30b0faf8965'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:14:02.128915 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ebc59589-1ce0-4b65-b24d-b30b0faf8965 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faf80adb087a441b9465e92d668fe6ad', 'request_id': 'req-ebc59589-1ce0-4b65-b24d-b30b0faf8965'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:14:02.134675 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ebc59589-1ce0-4b65-b24d-b30b0faf8965 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faf80adb087a441b9465e92d668fe6ad', 'request_id': 'req-ebc59589-1ce0-4b65-b24d-b30b0faf8965'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:14:02.197576 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ebc59589-1ce0-4b65-b24d-b30b0faf8965 admin admin] The token cache is being invalidate because project faf80adb087a441b9465e92d668fe6ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:14:02.197867 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ebc59589-1ce0-4b65-b24d-b30b0faf8965 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:14:02.198454 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-ebc59589-1ce0-4b65-b24d-b30b0faf8965 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:14:02.199976 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1433/2891] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:14:01 2022] DELETE /identity/v3/projects/faf80adb087a441b9465e92d668fe6ad => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:14:02.204662 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b01be6-6c26-4974-abac-af94bcfe7da6 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:14:02.214600 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b01be6-6c26-4974-abac-af94bcfe7da6 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:14:02.265869 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:14:02.272464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24ecda-a5b2-4d8e-a60d-feb710393b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:14:02.274318 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24ecda-a5b2-4d8e-a60d-feb710393b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:14:02.274318 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24ecda-a5b2-4d8e-a60d-feb710393b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:14:02.274318 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24ecda-a5b2-4d8e-a60d-feb710393b95 admin admin] PATH_INFO: `/v3/users/e05060b24cf542b4ae56f93cd340d9ed` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:14:02.274768 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24ecda-a5b2-4d8e-a60d-feb710393b95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e05060b24cf542b4ae56f93cd340d9ed)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:14:02.276704 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24ecda-a5b2-4d8e-a60d-feb710393b95 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:14:02.448036 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5e24ecda-a5b2-4d8e-a60d-feb710393b95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e05060b24cf542b4ae56f93cd340d9ed'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:14:02.465637 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5e24ecda-a5b2-4d8e-a60d-feb710393b95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e05060b24cf542b4ae56f93cd340d9ed'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:14:02.472938 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5e24ecda-a5b2-4d8e-a60d-feb710393b95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e05060b24cf542b4ae56f93cd340d9ed'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:14:02.473662 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5e24ecda-a5b2-4d8e-a60d-feb710393b95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e05060b24cf542b4ae56f93cd340d9ed'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:14:02.564247 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-5e24ecda-a5b2-4d8e-a60d-feb710393b95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e05060b24cf542b4ae56f93cd340d9ed'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:14:02.570654 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1459/2892] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:14:02 2022] DELETE /identity/v3/users/e05060b24cf542b4ae56f93cd340d9ed => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:14:02.988592 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc59589-1ce0-4b65-b24d-b30b0faf8965 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:14:02.990058 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ebc59589-1ce0-4b65-b24d-b30b0faf8965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:14:03.004393 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc59589-1ce0-4b65-b24d-b30b0faf8965 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:14:03.036974 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:14:03.063223 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a26b03-27c8-4f36-887a-e96a8ebcaefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:14:03.063842 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a26b03-27c8-4f36-887a-e96a8ebcaefc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:14:03.064119 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a26b03-27c8-4f36-887a-e96a8ebcaefc admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:14:03.064387 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a26b03-27c8-4f36-887a-e96a8ebcaefc admin admin] PATH_INFO: `/v3/projects/3ed17d4c7dec40afa9f073609c99784d` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:14:03.065481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a26b03-27c8-4f36-887a-e96a8ebcaefc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ed17d4c7dec40afa9f073609c99784d)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:14:03.067410 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a26b03-27c8-4f36-887a-e96a8ebcaefc admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:14:03.180176 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a1a26b03-27c8-4f36-887a-e96a8ebcaefc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ed17d4c7dec40afa9f073609c99784d', 'request_id': 'req-a1a26b03-27c8-4f36-887a-e96a8ebcaefc'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:14:03.186751 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a1a26b03-27c8-4f36-887a-e96a8ebcaefc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ed17d4c7dec40afa9f073609c99784d', 'request_id': 'req-a1a26b03-27c8-4f36-887a-e96a8ebcaefc'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:14:03.192361 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a1a26b03-27c8-4f36-887a-e96a8ebcaefc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ed17d4c7dec40afa9f073609c99784d', 'request_id': 'req-a1a26b03-27c8-4f36-887a-e96a8ebcaefc'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:14:03.258798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a1a26b03-27c8-4f36-887a-e96a8ebcaefc admin admin] The token cache is being invalidate because project 3ed17d4c7dec40afa9f073609c99784d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:14:03.259085 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a1a26b03-27c8-4f36-887a-e96a8ebcaefc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:14:03.259693 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-a1a26b03-27c8-4f36-887a-e96a8ebcaefc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:14:03.261145 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1434/2893] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:14:02 2022] DELETE /identity/v3/projects/3ed17d4c7dec40afa9f073609c99784d => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:14:03.266973 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803f3a4-eb1e-49fa-97b7-f4e8223d6ed9 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:14:03.267479 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803f3a4-eb1e-49fa-97b7-f4e8223d6ed9 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:14:03.267584 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803f3a4-eb1e-49fa-97b7-f4e8223d6ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:14:03.287193 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-3803f3a4-eb1e-49fa-97b7-f4e8223d6ed9 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:14:03.288314 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-3803f3a4-eb1e-49fa-97b7-f4e8223d6ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:14:03.349725 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1460/2894] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:14:03.358267 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f3dd0-a233-4455-a562-0759819c1cb3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:14:03.358550 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f3dd0-a233-4455-a562-0759819c1cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:14:03.358870 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f3dd0-a233-4455-a562-0759819c1cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:14:03.379573 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-ac6f3dd0-a233-4455-a562-0759819c1cb3 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:14:03.380757 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-ac6f3dd0-a233-4455-a562-0759819c1cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:14:03.388705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1435/2895] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:14:03.393058 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803f3a4-eb1e-49fa-97b7-f4e8223d6ed9 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:14:03.402568 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803f3a4-eb1e-49fa-97b7-f4e8223d6ed9 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:14:03.407395 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a28d09-17b0-4efa-8b9e-57264bf20157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:14:03.407981 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a28d09-17b0-4efa-8b9e-57264bf20157 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:14:03.408254 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a28d09-17b0-4efa-8b9e-57264bf20157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:14:03.408525 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a28d09-17b0-4efa-8b9e-57264bf20157 admin admin] PATH_INFO: `/v3/domains` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:14:03.409165 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a28d09-17b0-4efa-8b9e-57264bf20157 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81573) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:14:03.409464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a28d09-17b0-4efa-8b9e-57264bf20157 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:14:03.411093 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a28d09-17b0-4efa-8b9e-57264bf20157 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:14:03.423343 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1461/2896] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:14:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:14:03.427562 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f3dd0-a233-4455-a562-0759819c1cb3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:14:03.437346 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f3dd0-a233-4455-a562-0759819c1cb3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:14:03.442027 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c263020-1c09-4aa7-b0fe-3a9de4f9e12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:14:03.442613 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c263020-1c09-4aa7-b0fe-3a9de4f9e12f admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:14:03.442920 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c263020-1c09-4aa7-b0fe-3a9de4f9e12f admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:14:03.443187 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c263020-1c09-4aa7-b0fe-3a9de4f9e12f admin admin] PATH_INFO: `/v3/projects` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:14:03.443816 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c263020-1c09-4aa7-b0fe-3a9de4f9e12f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:14:03.445702 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c263020-1c09-4aa7-b0fe-3a9de4f9e12f admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:14:03.564864 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1436/2897] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:14:03 2022] POST /identity/v3/projects => generated 371 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:14:03.568911 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a28d09-17b0-4efa-8b9e-57264bf20157 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:14:03.578792 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a28d09-17b0-4efa-8b9e-57264bf20157 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:14:03.634872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334cf099-f5d8-40e3-8904-8ca4fc26c077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:14:03.635473 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cf099-f5d8-40e3-8904-8ca4fc26c077 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:14:03.635754 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cf099-f5d8-40e3-8904-8ca4fc26c077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:14:03.636029 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cf099-f5d8-40e3-8904-8ca4fc26c077 admin admin] PATH_INFO: `/v3/users` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:14:03.636692 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334cf099-f5d8-40e3-8904-8ca4fc26c077 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:14:03.639484 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334cf099-f5d8-40e3-8904-8ca4fc26c077 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:14:03.648744 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:14:03.745867 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1462/2898] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:14:03 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:14:03.750166 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c263020-1c09-4aa7-b0fe-3a9de4f9e12f admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:14:03.760371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c263020-1c09-4aa7-b0fe-3a9de4f9e12f admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:14:03.765461 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4926f-a89b-4002-9c71-342241000107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:14:03.766056 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4926f-a89b-4002-9c71-342241000107 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:14:03.766336 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4926f-a89b-4002-9c71-342241000107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:14:03.766605 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4926f-a89b-4002-9c71-342241000107 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:14:03.767300 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b4926f-a89b-4002-9c71-342241000107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:14:03.768925 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b4926f-a89b-4002-9c71-342241000107 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:14:03.779911 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1437/2899] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:14:03.783877 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334cf099-f5d8-40e3-8904-8ca4fc26c077 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:14:03.793813 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334cf099-f5d8-40e3-8904-8ca4fc26c077 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:14:03.801265 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74845bf-6c43-4ab6-9a28-2a413f05c351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:14:03.801867 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74845bf-6c43-4ab6-9a28-2a413f05c351 admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:14:03.803895 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74845bf-6c43-4ab6-9a28-2a413f05c351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:14:03.804407 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74845bf-6c43-4ab6-9a28-2a413f05c351 admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:14:03.805239 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74845bf-6c43-4ab6-9a28-2a413f05c351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:14:03.807137 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74845bf-6c43-4ab6-9a28-2a413f05c351 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:14:03.830501 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1463/2900] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:14:03.834899 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4926f-a89b-4002-9c71-342241000107 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:14:03.845529 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4926f-a89b-4002-9c71-342241000107 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:14:03.850105 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ac5ad-5db9-4794-9043-56961b5e7446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:14:03.854647 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ac5ad-5db9-4794-9043-56961b5e7446 admin admin] REQUEST_METHOD: `PUT` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:14:03.854647 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ac5ad-5db9-4794-9043-56961b5e7446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:14:03.854647 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ac5ad-5db9-4794-9043-56961b5e7446 admin admin] PATH_INFO: `/v3/projects/797df5a0a5704d1683176044bf78b1a8/users/ade06693efb44ac38e0c7bca08f0dbae/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:14:03.866646 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ac5ad-5db9-4794-9043-56961b5e7446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=797df5a0a5704d1683176044bf78b1a8, user_id=ade06693efb44ac38e0c7bca08f0dbae, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:14:03.869419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ac5ad-5db9-4794-9043-56961b5e7446 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:14:03.941220 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1438/2901] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:14:03 2022] PUT /identity/v3/projects/797df5a0a5704d1683176044bf78b1a8/users/ade06693efb44ac38e0c7bca08f0dbae/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:14:03.946233 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d68600-09d9-462d-96ec-67c5364033e1 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:14:03.946514 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d68600-09d9-462d-96ec-67c5364033e1 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:14:03.946841 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d68600-09d9-462d-96ec-67c5364033e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:14:03.967553 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-02d68600-09d9-462d-96ec-67c5364033e1 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:14:03.968856 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-02d68600-09d9-462d-96ec-67c5364033e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:14:04.027615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1464/2902] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:14:04.037999 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ac5ad-5db9-4794-9043-56961b5e7446 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:14:04.039833 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6e0ac5ad-5db9-4794-9043-56961b5e7446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:14:04.096365 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ac5ad-5db9-4794-9043-56961b5e7446 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:14:04.099926 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f507c7-a7ce-44fe-a961-b33fec276af1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:14:04.100521 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f507c7-a7ce-44fe-a961-b33fec276af1 service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:14:04.100798 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f507c7-a7ce-44fe-a961-b33fec276af1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:14:04.101066 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f507c7-a7ce-44fe-a961-b33fec276af1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:14:04.118877 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f507c7-a7ce-44fe-a961-b33fec276af1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:14:04.122154 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f507c7-a7ce-44fe-a961-b33fec276af1 service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:14:04.134513 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1439/2903] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:14:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:14:05.346907 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee4f93-be72-47ec-b4a4-af059ddcd7fa None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:14:05.347422 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee4f93-be72-47ec-b4a4-af059ddcd7fa None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:14:05.347545 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee4f93-be72-47ec-b4a4-af059ddcd7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:14:05.377155 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-98ee4f93-be72-47ec-b4a4-af059ddcd7fa None None] MFA Rules not processed for user `ade06693efb44ac38e0c7bca08f0dbae`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:14:05.379017 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-98ee4f93-be72-47ec-b4a4-af059ddcd7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:14:05.445731 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1465/2904] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:14:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:14:05.456928 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f507c7-a7ce-44fe-a961-b33fec276af1 service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:14:05.468865 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f2f507c7-a7ce-44fe-a961-b33fec276af1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:14:05.468865 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:14:05.517125 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f507c7-a7ce-44fe-a961-b33fec276af1 service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:14:05.520755 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ec9c23-386a-470e-b6bb-aed4d2824c51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:14:05.521373 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec9c23-386a-470e-b6bb-aed4d2824c51 service cinder] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:14:05.521647 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec9c23-386a-470e-b6bb-aed4d2824c51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:14:05.521913 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec9c23-386a-470e-b6bb-aed4d2824c51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:14:05.531318 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ec9c23-386a-470e-b6bb-aed4d2824c51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:14:05.532744 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ec9c23-386a-470e-b6bb-aed4d2824c51 service cinder] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:14:05.541602 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1440/2905] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:14:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:14:07.362206 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41d085-ed4b-4352-bff5-f5d809e4dadf None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:14:07.362728 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41d085-ed4b-4352-bff5-f5d809e4dadf None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:14:07.362833 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41d085-ed4b-4352-bff5-f5d809e4dadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:14:07.384435 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-5c41d085-ed4b-4352-bff5-f5d809e4dadf None None] MFA Rules not processed for user `ade06693efb44ac38e0c7bca08f0dbae`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:14:07.385640 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5c41d085-ed4b-4352-bff5-f5d809e4dadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:14:07.392678 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1466/2906] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:14:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:14:15.604528 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=25 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:14:15.925943 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f851ae-1b38-42bb-8d19-4cceb3e932c3 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:14:15.926221 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f851ae-1b38-42bb-8d19-4cceb3e932c3 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:14:15.926492 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f851ae-1b38-42bb-8d19-4cceb3e932c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:14:15.944976 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-b7f851ae-1b38-42bb-8d19-4cceb3e932c3 None None] MFA Rules not processed for user `aa4f534ee1ab4e639e99b9e4ebbe8363`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:14:15.946168 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b7f851ae-1b38-42bb-8d19-4cceb3e932c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:14:16.001814 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1441/2907] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:14:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:14:16.320108 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c41d085-ed4b-4352-bff5-f5d809e4dadf None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:14:16.321559 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-5c41d085-ed4b-4352-bff5-f5d809e4dadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:14:16.333875 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c41d085-ed4b-4352-bff5-f5d809e4dadf None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:14:16.337464 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8e508-2f67-47bd-abb7-ad7b25654403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:14:16.338057 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8e508-2f67-47bd-abb7-ad7b25654403 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:14:16.338331 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8e508-2f67-47bd-abb7-ad7b25654403 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:14:16.338628 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8e508-2f67-47bd-abb7-ad7b25654403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:14:16.340396 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c7c8e508-2f67-47bd-abb7-ad7b25654403 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:14:16.350830 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8e508-2f67-47bd-abb7-ad7b25654403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:14:16.351124 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8e508-2f67-47bd-abb7-ad7b25654403 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:14:16.359469 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1467/2908] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:14:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:14:25.994889 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=25 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:14:26.355406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:INSERT=2,keystone:SELECT=75 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:14:27.546335 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f851ae-1b38-42bb-8d19-4cceb3e932c3 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:14:27.547825 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-b7f851ae-1b38-42bb-8d19-4cceb3e932c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:14:27.624655 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f851ae-1b38-42bb-8d19-4cceb3e932c3 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:14:27.630131 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fbaf0-9830-46c1-9999-360381057a3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '636bfafb693044d283530063047fc0c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:14:27.630766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fbaf0-9830-46c1-9999-360381057a3e service nova] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:14:27.631047 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fbaf0-9830-46c1-9999-360381057a3e service nova] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:14:27.631310 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fbaf0-9830-46c1-9999-360381057a3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:14:27.633013 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-423fbaf0-9830-46c1-9999-360381057a3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:14:27.643761 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423fbaf0-9830-46c1-9999-360381057a3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:14:27.645371 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423fbaf0-9830-46c1-9999-360381057a3e service nova] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:14:27.654026 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1442/2909] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:14:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:14:28.022100 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f19c2-237a-463f-ae2d-d9456c8ab74d None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:14:28.022606 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f19c2-237a-463f-ae2d-d9456c8ab74d None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:14:28.022734 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f19c2-237a-463f-ae2d-d9456c8ab74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:14:28.041910 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-c42f19c2-237a-463f-ae2d-d9456c8ab74d None None] MFA Rules not processed for user `ade06693efb44ac38e0c7bca08f0dbae`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:14:28.043134 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c42f19c2-237a-463f-ae2d-d9456c8ab74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:14:28.050921 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1468/2910] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:14:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:14:37.650191 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=34 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:14:38.036775 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:14:51.952468 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d016281-6b16-4e61-8c37-60d9840e40f2 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:14:51.952958 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d016281-6b16-4e61-8c37-60d9840e40f2 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:14:51.953069 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d016281-6b16-4e61-8c37-60d9840e40f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:14:52.004142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-6d016281-6b16-4e61-8c37-60d9840e40f2 None None] MFA Rules not processed for user `b0a879085a194b0687e1cdcbafd48033`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:14:52.005337 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6d016281-6b16-4e61-8c37-60d9840e40f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:14:52.071749 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1443/2911] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Sep 11 00:14:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:14:52.207130 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f19c2-237a-463f-ae2d-d9456c8ab74d None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:14:52.208798 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-c42f19c2-237a-463f-ae2d-d9456c8ab74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:14:52.300484 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f19c2-237a-463f-ae2d-d9456c8ab74d None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:14:52.305523 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d23d1-8886-4de1-90f1-44ee9a53dd04 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '564e65585e9543e286b6041af870cf41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:14:52.306129 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d23d1-8886-4de1-90f1-44ee9a53dd04 service swift] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:14:52.306401 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d23d1-8886-4de1-90f1-44ee9a53dd04 service swift] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:14:52.306677 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d23d1-8886-4de1-90f1-44ee9a53dd04 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:14:52.314872 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1d23d1-8886-4de1-90f1-44ee9a53dd04 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:14:52.316433 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1d23d1-8886-4de1-90f1-44ee9a53dd04 service swift] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:14:52.324309 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1469/2912] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Sep 11 00:14:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 00:15:02.078642 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=40 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:15:02.321218 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=41 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:15:10.355131 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f859f0-c987-41f0-ab19-c6eef154b621 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:15:10.355606 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f859f0-c987-41f0-ab19-c6eef154b621 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:15:10.355718 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f859f0-c987-41f0-ab19-c6eef154b621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:15:10.376980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-02f859f0-c987-41f0-ab19-c6eef154b621 None None] MFA Rules not processed for user `b0a879085a194b0687e1cdcbafd48033`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:15:10.378193 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-02f859f0-c987-41f0-ab19-c6eef154b621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:15:10.385369 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1444/2913] 10.222.0.17 () {58 vars in 967 bytes} [Sun Sep 11 00:15:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:15:10.398300 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d23d1-8886-4de1-90f1-44ee9a53dd04 service swift] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:15:10.409019 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d23d1-8886-4de1-90f1-44ee9a53dd04 service swift] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:15:10.413995 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d58d69-c6d2-4067-8423-8a77a6d0696a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '564e65585e9543e286b6041af870cf41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:15:10.414587 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d58d69-c6d2-4067-8423-8a77a6d0696a service swift] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:15:10.414891 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d58d69-c6d2-4067-8423-8a77a6d0696a service swift] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:15:10.415159 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d58d69-c6d2-4067-8423-8a77a6d0696a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:15:10.424266 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d58d69-c6d2-4067-8423-8a77a6d0696a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:15:10.425849 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d58d69-c6d2-4067-8423-8a77a6d0696a service swift] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:15:10.434232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1470/2914] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Sep 11 00:15:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 00:15:10.741315 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f859f0-c987-41f0-ab19-c6eef154b621 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:15:10.742688 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-02f859f0-c987-41f0-ab19-c6eef154b621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:15:10.811896 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f859f0-c987-41f0-ab19-c6eef154b621 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:15:10.820882 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e45d8-63ed-49a3-aa39-a378fa5c43f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cb1d85a338e40f18e61887a1becb906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:15:10.820882 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e45d8-63ed-49a3-aa39-a378fa5c43f6 service placement] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:15:10.820882 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e45d8-63ed-49a3-aa39-a378fa5c43f6 service placement] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:15:10.820882 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e45d8-63ed-49a3-aa39-a378fa5c43f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:15:10.821406 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-421e45d8-63ed-49a3-aa39-a378fa5c43f6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:15:10.831932 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e45d8-63ed-49a3-aa39-a378fa5c43f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:15:10.833800 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e45d8-63ed-49a3-aa39-a378fa5c43f6 service placement] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:15:10.844253 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1445/2915] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:15:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:15:13.310630 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d58d69-c6d2-4067-8423-8a77a6d0696a service swift] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:15:13.322316 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d58d69-c6d2-4067-8423-8a77a6d0696a service swift] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:15:13.327289 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a8d38d-202e-46d3-b0da-cc170f05c55a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cb1d85a338e40f18e61887a1becb906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:15:13.328111 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a8d38d-202e-46d3-b0da-cc170f05c55a service placement] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:15:13.328512 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a8d38d-202e-46d3-b0da-cc170f05c55a service placement] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:15:13.328784 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a8d38d-202e-46d3-b0da-cc170f05c55a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:15:13.331183 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-41a8d38d-202e-46d3-b0da-cc170f05c55a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:15:13.343991 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a8d38d-202e-46d3-b0da-cc170f05c55a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:15:13.345647 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a8d38d-202e-46d3-b0da-cc170f05c55a service placement] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:15:13.355796 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1471/2916] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:15:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:15:13.438033 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e45d8-63ed-49a3-aa39-a378fa5c43f6 service placement] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:15:13.448807 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e45d8-63ed-49a3-aa39-a378fa5c43f6 service placement] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:15:13.454092 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053aedd-1284-426c-98fb-c9e3dc2f853a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:15:13.454779 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053aedd-1284-426c-98fb-c9e3dc2f853a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:15:13.455065 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053aedd-1284-426c-98fb-c9e3dc2f853a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:15:13.455369 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053aedd-1284-426c-98fb-c9e3dc2f853a admin admin] PATH_INFO: `/v3/users/ade06693efb44ac38e0c7bca08f0dbae` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:15:13.456491 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f053aedd-1284-426c-98fb-c9e3dc2f853a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ade06693efb44ac38e0c7bca08f0dbae)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:15:13.458694 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f053aedd-1284-426c-98fb-c9e3dc2f853a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:15:13.652279 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f053aedd-1284-426c-98fb-c9e3dc2f853a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ade06693efb44ac38e0c7bca08f0dbae'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:15:13.660189 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f053aedd-1284-426c-98fb-c9e3dc2f853a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ade06693efb44ac38e0c7bca08f0dbae'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:15:13.672359 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f053aedd-1284-426c-98fb-c9e3dc2f853a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ade06693efb44ac38e0c7bca08f0dbae'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:15:13.762999 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f053aedd-1284-426c-98fb-c9e3dc2f853a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ade06693efb44ac38e0c7bca08f0dbae'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:15:13.763608 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-f053aedd-1284-426c-98fb-c9e3dc2f853a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ade06693efb44ac38e0c7bca08f0dbae'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:15:13.770791 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1446/2917] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:15:13 2022] DELETE /identity/v3/users/ade06693efb44ac38e0c7bca08f0dbae => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:15:14.044438 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a8d38d-202e-46d3-b0da-cc170f05c55a service placement] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:15:14.055860 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a8d38d-202e-46d3-b0da-cc170f05c55a service placement] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:15:14.113831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea13f02-694c-416b-8a72-573e42af64d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:15:14.114447 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea13f02-694c-416b-8a72-573e42af64d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:15:14.114755 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea13f02-694c-416b-8a72-573e42af64d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:15:14.115029 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea13f02-694c-416b-8a72-573e42af64d5 admin admin] PATH_INFO: `/v3/projects/797df5a0a5704d1683176044bf78b1a8` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:15:14.116117 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea13f02-694c-416b-8a72-573e42af64d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=797df5a0a5704d1683176044bf78b1a8)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:15:14.118019 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea13f02-694c-416b-8a72-573e42af64d5 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:15:14.249011 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1ea13f02-694c-416b-8a72-573e42af64d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '797df5a0a5704d1683176044bf78b1a8', 'request_id': 'req-1ea13f02-694c-416b-8a72-573e42af64d5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:15:14.317045 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1ea13f02-694c-416b-8a72-573e42af64d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '797df5a0a5704d1683176044bf78b1a8', 'request_id': 'req-1ea13f02-694c-416b-8a72-573e42af64d5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:15:14.323087 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1ea13f02-694c-416b-8a72-573e42af64d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '797df5a0a5704d1683176044bf78b1a8', 'request_id': 'req-1ea13f02-694c-416b-8a72-573e42af64d5'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:15:14.335303 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1ea13f02-694c-416b-8a72-573e42af64d5 admin admin] The token cache is being invalidate because project 797df5a0a5704d1683176044bf78b1a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:15:14.341299 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1ea13f02-694c-416b-8a72-573e42af64d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:15:14.341299 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-1ea13f02-694c-416b-8a72-573e42af64d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:15:14.341299 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1472/2918] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:15:14 2022] DELETE /identity/v3/projects/797df5a0a5704d1683176044bf78b1a8 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:15:14.342554 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0898d1-39de-44cf-9f36-62ea009b9dc7 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:15:14.342876 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0898d1-39de-44cf-9f36-62ea009b9dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:15:14.343146 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0898d1-39de-44cf-9f36-62ea009b9dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:15:14.362795 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-6d0898d1-39de-44cf-9f36-62ea009b9dc7 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:15:14.363988 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-6d0898d1-39de-44cf-9f36-62ea009b9dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:15:14.410201 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:15:14.421036 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1447/2919] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:15:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:15:14.434406 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551dc99-01d6-4e5e-9ff7-0fcd5772fc12 None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:15:14.434714 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551dc99-01d6-4e5e-9ff7-0fcd5772fc12 None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:15:14.434988 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551dc99-01d6-4e5e-9ff7-0fcd5772fc12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:15:14.455141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-2551dc99-01d6-4e5e-9ff7-0fcd5772fc12 None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:15:14.456318 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2551dc99-01d6-4e5e-9ff7-0fcd5772fc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:15:14.464312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1473/2920] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:15:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:15:14.468954 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0898d1-39de-44cf-9f36-62ea009b9dc7 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:15:14.484611 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0898d1-39de-44cf-9f36-62ea009b9dc7 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:15:14.489871 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7da7e2-99eb-4b36-8254-7d7180a0afe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:15:14.490502 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7da7e2-99eb-4b36-8254-7d7180a0afe8 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:15:14.490826 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7da7e2-99eb-4b36-8254-7d7180a0afe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:15:14.491098 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7da7e2-99eb-4b36-8254-7d7180a0afe8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:15:14.491734 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7da7e2-99eb-4b36-8254-7d7180a0afe8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:15:14.492031 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7da7e2-99eb-4b36-8254-7d7180a0afe8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:15:14.493654 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7da7e2-99eb-4b36-8254-7d7180a0afe8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:15:14.505588 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1448/2921] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Sep 11 00:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:15:14.510022 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551dc99-01d6-4e5e-9ff7-0fcd5772fc12 None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:15:14.519398 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551dc99-01d6-4e5e-9ff7-0fcd5772fc12 None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:15:14.524082 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e491a-f421-4507-98fa-dc5dd97ec983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:15:14.524672 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e491a-f421-4507-98fa-dc5dd97ec983 admin admin] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:15:14.524955 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e491a-f421-4507-98fa-dc5dd97ec983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:15:14.525227 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e491a-f421-4507-98fa-dc5dd97ec983 admin admin] PATH_INFO: `/v3/projects` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:15:14.525863 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e491a-f421-4507-98fa-dc5dd97ec983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:15:14.527807 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e491a-f421-4507-98fa-dc5dd97ec983 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:15:14.639026 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1474/2922] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Sep 11 00:15:14 2022] POST /identity/v3/projects => generated 391 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:15:14.643765 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7da7e2-99eb-4b36-8254-7d7180a0afe8 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:15:14.654334 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7da7e2-99eb-4b36-8254-7d7180a0afe8 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:15:14.720617 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c2893-6dfa-42ab-ba37-29a4d7312b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:15:14.721227 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c2893-6dfa-42ab-ba37-29a4d7312b0a admin admin] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:15:14.721504 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c2893-6dfa-42ab-ba37-29a4d7312b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:15:14.721776 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c2893-6dfa-42ab-ba37-29a4d7312b0a admin admin] PATH_INFO: `/v3/users` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:15:14.722419 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c2893-6dfa-42ab-ba37-29a4d7312b0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:15:14.724442 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c2893-6dfa-42ab-ba37-29a4d7312b0a admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:15:14.844331 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1449/2923] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Sep 11 00:15:14 2022] POST /identity/v3/users => generated 296 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:15:14.848547 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e491a-f421-4507-98fa-dc5dd97ec983 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:15:14.859546 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e491a-f421-4507-98fa-dc5dd97ec983 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:15:14.864851 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ecbafe-f8c0-4a7a-a927-d6386df73beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:15:14.865547 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ecbafe-f8c0-4a7a-a927-d6386df73beb admin admin] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:15:14.865822 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ecbafe-f8c0-4a7a-a927-d6386df73beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:15:14.866095 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ecbafe-f8c0-4a7a-a927-d6386df73beb admin admin] PATH_INFO: `/v3/roles` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:15:14.867153 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ecbafe-f8c0-4a7a-a927-d6386df73beb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:15:14.868373 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ecbafe-f8c0-4a7a-a927-d6386df73beb admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:15:14.879576 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1475/2924] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:15:14.883438 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c2893-6dfa-42ab-ba37-29a4d7312b0a admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:15:14.894705 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c2893-6dfa-42ab-ba37-29a4d7312b0a admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:15:14.899516 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c063ad-e8fe-4900-8366-6ee6ff64cc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:15:14.900113 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c063ad-e8fe-4900-8366-6ee6ff64cc89 admin admin] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:15:14.900394 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c063ad-e8fe-4900-8366-6ee6ff64cc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:15:14.900662 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c063ad-e8fe-4900-8366-6ee6ff64cc89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:15:14.901302 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c063ad-e8fe-4900-8366-6ee6ff64cc89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:15:14.902929 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c063ad-e8fe-4900-8366-6ee6ff64cc89 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:15:14.915120 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1450/2925] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Sep 11 00:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:15:14.919156 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ecbafe-f8c0-4a7a-a927-d6386df73beb admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:15:14.929648 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ecbafe-f8c0-4a7a-a927-d6386df73beb admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:15:14.934155 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cca7f2-9259-4cc3-9cf5-e136c189e5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:15:14.934829 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cca7f2-9259-4cc3-9cf5-e136c189e5c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:15:14.935106 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cca7f2-9259-4cc3-9cf5-e136c189e5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:15:14.935382 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cca7f2-9259-4cc3-9cf5-e136c189e5c6 admin admin] PATH_INFO: `/v3/projects/401e82ae0e794caaa6031081413acfc3/users/52079c5fd92e4b5cb8e8cb99a467329f/roles/824f5fe1f02b447382190a8c236c27db` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:15:14.951814 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cca7f2-9259-4cc3-9cf5-e136c189e5c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=401e82ae0e794caaa6031081413acfc3, user_id=52079c5fd92e4b5cb8e8cb99a467329f, role_id=824f5fe1f02b447382190a8c236c27db)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:15:14.955655 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cca7f2-9259-4cc3-9cf5-e136c189e5c6 admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:15:15.063083 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:15:15.071845 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1476/2926] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Sep 11 00:15:14 2022] PUT /identity/v3/projects/401e82ae0e794caaa6031081413acfc3/users/52079c5fd92e4b5cb8e8cb99a467329f/roles/824f5fe1f02b447382190a8c236c27db => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:15:15.079275 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0fe0f-2b62-45cc-9520-b9bc474aec96 None None] REQUEST_METHOD: `POST` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:15:15.079585 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0fe0f-2b62-45cc-9520-b9bc474aec96 None None] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:15:15.079854 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0fe0f-2b62-45cc-9520-b9bc474aec96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:15:15.132240 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.auth.core [None req-dcc0fe0f-2b62-45cc-9520-b9bc474aec96 None None] MFA Rules not processed for user `52079c5fd92e4b5cb8e8cb99a467329f`. Rule list: `[]` (Enabled: `True`). {{(pid=81574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:15:15.132240 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-dcc0fe0f-2b62-45cc-9520-b9bc474aec96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:15:15.175142 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1451/2927] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:15:15 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:15:15.186453 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cca7f2-9259-4cc3-9cf5-e136c189e5c6 admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:15:15.188969 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-10cca7f2-9259-4cc3-9cf5-e136c189e5c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:15:15.247186 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cca7f2-9259-4cc3-9cf5-e136c189e5c6 admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:15:15.251931 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e57d8-00e0-4387-b73f-485513d7421b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6fccf47e1ec4324983c5218ec3c96a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:15:15.252542 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e57d8-00e0-4387-b73f-485513d7421b service cinder] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:15:15.252817 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e57d8-00e0-4387-b73f-485513d7421b service cinder] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:15:15.253086 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e57d8-00e0-4387-b73f-485513d7421b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:15:15.261563 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e57d8-00e0-4387-b73f-485513d7421b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:15:15.263014 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e57d8-00e0-4387-b73f-485513d7421b service cinder] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:15:15.274257 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1477/2928] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:15:15 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:15:25.171550 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=74,keystone:INSERT=3 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:15:25.273498 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=26 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:15:36.304281 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0fe0f-2b62-45cc-9520-b9bc474aec96 None None] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:15:36.373059 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0fe0f-2b62-45cc-9520-b9bc474aec96 None None] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:15:36.383529 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852d420-2a97-43c1-94c4-41d128eb395e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:15:36.384153 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852d420-2a97-43c1-94c4-41d128eb395e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:15:36.384425 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852d420-2a97-43c1-94c4-41d128eb395e admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:15:36.384692 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852d420-2a97-43c1-94c4-41d128eb395e admin admin] PATH_INFO: `/v3/users/52079c5fd92e4b5cb8e8cb99a467329f` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:15:36.385794 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852d420-2a97-43c1-94c4-41d128eb395e admin admin] RBAC: Authorizing `identity:delete_user(user_id=52079c5fd92e4b5cb8e8cb99a467329f)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:15:36.387754 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852d420-2a97-43c1-94c4-41d128eb395e admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:15:36.541427 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7852d420-2a97-43c1-94c4-41d128eb395e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52079c5fd92e4b5cb8e8cb99a467329f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:15:36.548242 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7852d420-2a97-43c1-94c4-41d128eb395e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52079c5fd92e4b5cb8e8cb99a467329f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:15:36.560735 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7852d420-2a97-43c1-94c4-41d128eb395e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52079c5fd92e4b5cb8e8cb99a467329f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:15:36.642404 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7852d420-2a97-43c1-94c4-41d128eb395e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52079c5fd92e4b5cb8e8cb99a467329f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:15:36.643041 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-7852d420-2a97-43c1-94c4-41d128eb395e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52079c5fd92e4b5cb8e8cb99a467329f'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:15:36.649058 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1452/2929] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:15:36 2022] DELETE /identity/v3/users/52079c5fd92e4b5cb8e8cb99a467329f => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:15:36.653743 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff87635-e079-4645-bc48-03e4f00964be None None] REQUEST_METHOD: `POST` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:15:36.654231 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff87635-e079-4645-bc48-03e4f00964be None None] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:15:36.654355 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff87635-e079-4645-bc48-03e4f00964be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:15:36.674476 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.auth.core [None req-aff87635-e079-4645-bc48-03e4f00964be None None] MFA Rules not processed for user `5243f89b8bf44f868ec5406d662b5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81573) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:15:36.676911 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-aff87635-e079-4645-bc48-03e4f00964be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:15:36.741807 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1478/2930] 10.222.0.17 () {58 vars in 958 bytes} [Sun Sep 11 00:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:15:36.751563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852d420-2a97-43c1-94c4-41d128eb395e admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:15:36.753776 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-7852d420-2a97-43c1-94c4-41d128eb395e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:15:36.810863 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852d420-2a97-43c1-94c4-41d128eb395e admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:15:36.814563 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f4436-1a34-4c78-afbf-3c8c756d7abd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:15:36.824139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f4436-1a34-4c78-afbf-3c8c756d7abd service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:15:36.824139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f4436-1a34-4c78-afbf-3c8c756d7abd service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:15:36.824139 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f4436-1a34-4c78-afbf-3c8c756d7abd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:15:36.826044 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4f4436-1a34-4c78-afbf-3c8c756d7abd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:15:36.827526 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4f4436-1a34-4c78-afbf-3c8c756d7abd service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:15:36.837467 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1453/2931] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:15:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:15:37.266902 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff87635-e079-4645-bc48-03e4f00964be None None] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:15:37.268340 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-aff87635-e079-4645-bc48-03e4f00964be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:15:37.281199 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff87635-e079-4645-bc48-03e4f00964be None None] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:15:37.287192 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723a8b5-0c7b-4462-8458-05cd6ffb11db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:15:37.287850 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723a8b5-0c7b-4462-8458-05cd6ffb11db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:15:37.288120 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723a8b5-0c7b-4462-8458-05cd6ffb11db admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:15:37.288386 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723a8b5-0c7b-4462-8458-05cd6ffb11db admin admin] PATH_INFO: `/v3/projects/401e82ae0e794caaa6031081413acfc3` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:15:37.289571 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4723a8b5-0c7b-4462-8458-05cd6ffb11db admin admin] RBAC: Authorizing `identity:delete_project(project_id=401e82ae0e794caaa6031081413acfc3)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:15:37.291890 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4723a8b5-0c7b-4462-8458-05cd6ffb11db admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:15:37.391036 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4723a8b5-0c7b-4462-8458-05cd6ffb11db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '401e82ae0e794caaa6031081413acfc3', 'request_id': 'req-4723a8b5-0c7b-4462-8458-05cd6ffb11db'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:15:37.465207 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4723a8b5-0c7b-4462-8458-05cd6ffb11db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '401e82ae0e794caaa6031081413acfc3', 'request_id': 'req-4723a8b5-0c7b-4462-8458-05cd6ffb11db'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:15:37.471069 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4723a8b5-0c7b-4462-8458-05cd6ffb11db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '401e82ae0e794caaa6031081413acfc3', 'request_id': 'req-4723a8b5-0c7b-4462-8458-05cd6ffb11db'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:15:37.482859 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4723a8b5-0c7b-4462-8458-05cd6ffb11db admin admin] The token cache is being invalidate because project 401e82ae0e794caaa6031081413acfc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81573) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:15:37.483141 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4723a8b5-0c7b-4462-8458-05cd6ffb11db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:15:37.483762 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-4723a8b5-0c7b-4462-8458-05cd6ffb11db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:15:37.485230 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1479/2932] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:15:37 2022] DELETE /identity/v3/projects/401e82ae0e794caaa6031081413acfc3 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:15:46.832815 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:15:47.479716 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:16:21.381890 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f4436-1a34-4c78-afbf-3c8c756d7abd service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:16:21.391016 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f4436-1a34-4c78-afbf-3c8c756d7abd service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:16:21.440838 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945c6dc-9dc6-4bad-834e-40d728a39f6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:16:21.441445 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945c6dc-9dc6-4bad-834e-40d728a39f6c service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:16:21.441724 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945c6dc-9dc6-4bad-834e-40d728a39f6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:16:21.441995 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945c6dc-9dc6-4bad-834e-40d728a39f6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:16:21.443789 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9945c6dc-9dc6-4bad-834e-40d728a39f6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:16:21.516980 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9945c6dc-9dc6-4bad-834e-40d728a39f6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:16:21.518423 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9945c6dc-9dc6-4bad-834e-40d728a39f6c service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:16:21.530294 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1454/2933] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:16:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:16:22.729671 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723a8b5-0c7b-4462-8458-05cd6ffb11db admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:16:22.731312 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-4723a8b5-0c7b-4462-8458-05cd6ffb11db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:16:22.743744 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723a8b5-0c7b-4462-8458-05cd6ffb11db admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:16:22.747232 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314af246-07c1-4fd5-b5b9-25171c9db7b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:16:22.748586 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314af246-07c1-4fd5-b5b9-25171c9db7b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:16:22.748586 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314af246-07c1-4fd5-b5b9-25171c9db7b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:16:22.748899 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314af246-07c1-4fd5-b5b9-25171c9db7b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:16:22.750511 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-314af246-07c1-4fd5-b5b9-25171c9db7b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:16:22.764997 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314af246-07c1-4fd5-b5b9-25171c9db7b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:16:22.766418 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314af246-07c1-4fd5-b5b9-25171c9db7b0 service neutron] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:16:22.777573 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1480/2934] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:16:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:16:31.524401 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=54 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:16:32.772444 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=6 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:16:37.202644 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945c6dc-9dc6-4bad-834e-40d728a39f6c service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:16:37.204473 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-9945c6dc-9dc6-4bad-834e-40d728a39f6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:16:37.217234 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945c6dc-9dc6-4bad-834e-40d728a39f6c service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:16:37.223966 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9276cd8-0395-45c0-ae81-a7768b4626dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53e79d9fb2cd452d8d999a84b6425959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:16:37.224570 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9276cd8-0395-45c0-ae81-a7768b4626dd service neutron] REQUEST_METHOD: `GET` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:16:37.224844 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9276cd8-0395-45c0-ae81-a7768b4626dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:16:37.225110 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9276cd8-0395-45c0-ae81-a7768b4626dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:16:37.226888 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-f9276cd8-0395-45c0-ae81-a7768b4626dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:16:37.292959 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9276cd8-0395-45c0-ae81-a7768b4626dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:16:37.294421 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9276cd8-0395-45c0-ae81-a7768b4626dd service neutron] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:16:37.305796 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1455/2935] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:16:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:16:44.630705 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314af246-07c1-4fd5-b5b9-25171c9db7b0 service neutron] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:16:44.630705 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-314af246-07c1-4fd5-b5b9-25171c9db7b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:16:44.643686 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314af246-07c1-4fd5-b5b9-25171c9db7b0 service neutron] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:16:44.648265 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a832e-4be9-4663-a067-e72ea1e82f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:16:44.648885 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a832e-4be9-4663-a067-e72ea1e82f5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:16:44.649156 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a832e-4be9-4663-a067-e72ea1e82f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:16:44.649417 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a832e-4be9-4663-a067-e72ea1e82f5a admin admin] PATH_INFO: `/v3/users/aa4f534ee1ab4e639e99b9e4ebbe8363` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:16:44.650515 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a832e-4be9-4663-a067-e72ea1e82f5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa4f534ee1ab4e639e99b9e4ebbe8363)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:16:44.652487 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a832e-4be9-4663-a067-e72ea1e82f5a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:16:44.872530 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2a6a832e-4be9-4663-a067-e72ea1e82f5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa4f534ee1ab4e639e99b9e4ebbe8363'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:16:44.884763 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2a6a832e-4be9-4663-a067-e72ea1e82f5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa4f534ee1ab4e639e99b9e4ebbe8363'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:16:44.891578 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2a6a832e-4be9-4663-a067-e72ea1e82f5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa4f534ee1ab4e639e99b9e4ebbe8363'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:16:44.893138 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2a6a832e-4be9-4663-a067-e72ea1e82f5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa4f534ee1ab4e639e99b9e4ebbe8363'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:16:45.029925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-2a6a832e-4be9-4663-a067-e72ea1e82f5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa4f534ee1ab4e639e99b9e4ebbe8363'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:16:45.037506 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1481/2936] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:16:44 2022] DELETE /identity/v3/users/aa4f534ee1ab4e639e99b9e4ebbe8363 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:16:45.305857 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9276cd8-0395-45c0-ae81-a7768b4626dd service neutron] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:16:45.320336 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9276cd8-0395-45c0-ae81-a7768b4626dd service neutron] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:16:45.380488 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e582f8-fc1a-4ed8-ab5d-7360924803f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:16:45.381140 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e582f8-fc1a-4ed8-ab5d-7360924803f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:16:45.381429 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e582f8-fc1a-4ed8-ab5d-7360924803f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:16:45.381685 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e582f8-fc1a-4ed8-ab5d-7360924803f8 admin admin] PATH_INFO: `/v3/projects/a819c981f44e4d21ab8b8289d16c19fb` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:16:45.382853 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e582f8-fc1a-4ed8-ab5d-7360924803f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a819c981f44e4d21ab8b8289d16c19fb)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:16:45.384766 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e582f8-fc1a-4ed8-ab5d-7360924803f8 admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:16:45.510809 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-67e582f8-fc1a-4ed8-ab5d-7360924803f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a819c981f44e4d21ab8b8289d16c19fb', 'request_id': 'req-67e582f8-fc1a-4ed8-ab5d-7360924803f8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:16:45.517208 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-67e582f8-fc1a-4ed8-ab5d-7360924803f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a819c981f44e4d21ab8b8289d16c19fb', 'request_id': 'req-67e582f8-fc1a-4ed8-ab5d-7360924803f8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:16:45.522842 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-67e582f8-fc1a-4ed8-ab5d-7360924803f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a819c981f44e4d21ab8b8289d16c19fb', 'request_id': 'req-67e582f8-fc1a-4ed8-ab5d-7360924803f8'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:16:45.589358 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-67e582f8-fc1a-4ed8-ab5d-7360924803f8 admin admin] The token cache is being invalidate because project a819c981f44e4d21ab8b8289d16c19fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:16:45.589640 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-67e582f8-fc1a-4ed8-ab5d-7360924803f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:16:45.590248 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-67e582f8-fc1a-4ed8-ab5d-7360924803f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:16:45.591750 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1456/2937] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:16:45 2022] DELETE /identity/v3/projects/a819c981f44e4d21ab8b8289d16c19fb => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:16:45.595831 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a832e-4be9-4663-a067-e72ea1e82f5a admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:16:45.600615 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-2a6a832e-4be9-4663-a067-e72ea1e82f5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:16:45.610263 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a832e-4be9-4663-a067-e72ea1e82f5a admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:16:45.666672 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4376ce2-0d16-4b55-8e7e-87df19c77d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:16:45.667368 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4376ce2-0d16-4b55-8e7e-87df19c77d3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:16:45.667630 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4376ce2-0d16-4b55-8e7e-87df19c77d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:16:45.667925 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4376ce2-0d16-4b55-8e7e-87df19c77d3a admin admin] PATH_INFO: `/v3/users/73592f21d8d44b0dba097094351e37bb` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:16:45.669063 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4376ce2-0d16-4b55-8e7e-87df19c77d3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=73592f21d8d44b0dba097094351e37bb)` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:16:45.671071 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4376ce2-0d16-4b55-8e7e-87df19c77d3a admin admin] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:16:45.879064 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-a4376ce2-0d16-4b55-8e7e-87df19c77d3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73592f21d8d44b0dba097094351e37bb'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:16:45.891136 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-a4376ce2-0d16-4b55-8e7e-87df19c77d3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73592f21d8d44b0dba097094351e37bb'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:16:45.898039 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-a4376ce2-0d16-4b55-8e7e-87df19c77d3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73592f21d8d44b0dba097094351e37bb'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:16:45.898738 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-a4376ce2-0d16-4b55-8e7e-87df19c77d3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73592f21d8d44b0dba097094351e37bb'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:16:45.900385 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:16:45.984115 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.notifications [None req-a4376ce2-0d16-4b55-8e7e-87df19c77d3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73592f21d8d44b0dba097094351e37bb'} {{(pid=81573) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:16:45.990916 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1482/2938] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Sep 11 00:16:45 2022] DELETE /identity/v3/users/73592f21d8d44b0dba097094351e37bb => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:16:46.254700 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e582f8-fc1a-4ed8-ab5d-7360924803f8 admin admin] Authenticating user token {{(pid=81574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:16:46.256157 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.fernet_utils [None req-67e582f8-fc1a-4ed8-ab5d-7360924803f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:16:46.270512 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e582f8-fc1a-4ed8-ab5d-7360924803f8 admin admin] Validating token access rules against request {{(pid=81574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:16:46.292040 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:16:46.330590 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863b84f-a728-4cf5-bd74-e54fdba7758c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5243f89b8bf44f868ec5406d662b5cd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab88f917c063407982ac832bef946248', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:16:46.334128 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863b84f-a728-4cf5-bd74-e54fdba7758c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:16:46.334128 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863b84f-a728-4cf5-bd74-e54fdba7758c admin admin] SCRIPT_NAME: `/identity` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:16:46.334128 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863b84f-a728-4cf5-bd74-e54fdba7758c admin admin] PATH_INFO: `/v3/projects/1fc676ca5ade41519650a1048e1eb544` {{(pid=81574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:16:46.334128 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c863b84f-a728-4cf5-bd74-e54fdba7758c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fc676ca5ade41519650a1048e1eb544)` {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:16:46.335604 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c863b84f-a728-4cf5-bd74-e54fdba7758c admin admin] RBAC: Authorization granted {{(pid=81574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:16:46.452792 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c863b84f-a728-4cf5-bd74-e54fdba7758c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fc676ca5ade41519650a1048e1eb544', 'request_id': 'req-c863b84f-a728-4cf5-bd74-e54fdba7758c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:16:46.459429 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c863b84f-a728-4cf5-bd74-e54fdba7758c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fc676ca5ade41519650a1048e1eb544', 'request_id': 'req-c863b84f-a728-4cf5-bd74-e54fdba7758c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:16:46.464922 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c863b84f-a728-4cf5-bd74-e54fdba7758c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fc676ca5ade41519650a1048e1eb544', 'request_id': 'req-c863b84f-a728-4cf5-bd74-e54fdba7758c'} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:16:46.538248 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c863b84f-a728-4cf5-bd74-e54fdba7758c admin admin] The token cache is being invalidate because project 1fc676ca5ade41519650a1048e1eb544 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:16:46.538481 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c863b84f-a728-4cf5-bd74-e54fdba7758c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:16:46.539084 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG keystone.notifications [None req-c863b84f-a728-4cf5-bd74-e54fdba7758c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:16:46.540571 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: [pid: 81574|app: 0|req: 1457/2939] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Sep 11 00:16:46 2022] DELETE /identity/v3/projects/1fc676ca5ade41519650a1048e1eb544 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:16:56.155823 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:16:56.535022 dv-cinder-852009-6-iscsi devstack@keystone.service[81574]: DEBUG dbcounter [-] [81574] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:20:15.703699 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4376ce2-0d16-4b55-8e7e-87df19c77d3a admin admin] Authenticating user token {{(pid=81573) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:20:15.705230 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-a4376ce2-0d16-4b55-8e7e-87df19c77d3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:20:15.846766 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4376ce2-0d16-4b55-8e7e-87df19c77d3a admin admin] Validating token access rules against request {{(pid=81573) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:20:15.904795 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1dbde-64b4-4b6e-94d9-178ee935e2bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cb1d85a338e40f18e61887a1becb906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '947277c4458645899287e6818947c076', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81573) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:20:15.905399 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1dbde-64b4-4b6e-94d9-178ee935e2bc service placement] REQUEST_METHOD: `GET` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:20:15.905680 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1dbde-64b4-4b6e-94d9-178ee935e2bc service placement] SCRIPT_NAME: `/identity` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:20:15.906530 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1dbde-64b4-4b6e-94d9-178ee935e2bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81573) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:20:15.908352 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.fernet_utils [None req-ecb1dbde-64b4-4b6e-94d9-178ee935e2bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81573) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:20:15.919823 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1dbde-64b4-4b6e-94d9-178ee935e2bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:20:15.924306 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1dbde-64b4-4b6e-94d9-178ee935e2bc service placement] RBAC: Authorization granted {{(pid=81573) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:20:15.931404 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: [pid: 81573|app: 0|req: 1483/2940] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Sep 11 00:20:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:20:25.927145 dv-cinder-852009-6-iscsi devstack@keystone.service[81573]: DEBUG dbcounter [-] [81573] Writing DB stats keystone:SELECT=89 {{(pid=81573) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}